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ABSTRACT 



This monograph is part of a series on institutional support 
for student assessment. This report, which covers the second stage of the 
study, analyzes the results of a national survey of postsecondary education 
institutions that examined how institutions approach student assessment, 
their patterns of organizational and administrative support, and the uses and 
impacts of these efforts, focusing on undergraduate education. The survey 
instrument was based on a literature review and conceptual framework 
developed during the first stage. The survey population was all public and 
private postsecondary institutions that offer undergraduate programs at the 
associate or baccalaureate level . Analysis of the data suggest that student 
assessment is becoming a common practice in the academic management of U.S. 
postsecondary education, but it is not yet institutionalized or deeply 
embedded in institutional support patterns, policies, and practices. The 
researchers also conclude that the conceptual framework used appears useful, 
and that the survey instrument provides a checklist for institutions to 
examine their assessment approach. Appendixes include the survey instrument, 
the "Inventory of Institutional Support for Student Assessment"; 
correspondence related to the Inventory; institutional response rates by 
state,* factor analysis results,* and related reports, publications, and 
presentations. (Contains approximately 155 references.) (CH) 
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1. Overview of the Study 



1 . 1 Purpose, Rationale, and Nature of the Study 
Over the past decade assessment and improvement of student performance has been the 
focus of much discussion and many efforts both within and external to colleges and universities. 
During that time there has been a progressive increase in the number of postsecondary ins titutions 
engaged in some form of student assessment (El-Khawas, 1990, 1995). A considerable amount of 
faculty and administrative time and effort has been invested in promoting, supporting and 
implementing student assessment. To date there has been little systematic examination of the 
institutional responses to external demands, the institutional approaches to student assessment, and 
the organizational and administrative patterns that are formulated to promote and implement student 
assessment. Even less available is evidence regarding the institutional use and impact of st ude nt 
assessment (Banta, Lund, Black, & Oblander, 1996; Ewell, 1988b, 1997; Gray & Banta, 1997). 
Postsecondaiy institutions throughout the nation continue to search for appropriate and effective 
strategies for student assessment but have little credible evidence to guide their efforts. 

NCPI Research Project 5.2 on “Institutional Support for Enhancing Student Assessment 
and Performance” addresses this gap by examining how postsecondaiy institutions respond to 
external pressures for and promote the effective use of student assessment practices that imp act 
student learning and institutional performance. For the purposes of this research, student 
assessment refers to those activities focused on measuring dimensions of student performance 
other than traditional end-of-course grading. 

Building on a review of the literature and a conceptual framework developed during the 
first stage of our research on institutional support for student assessment, this study reports on the 
second stage of our research: the design, implementation and analysis of a national survey of 
institutional approaches to student assessment, the patterns of organizational and administrative 
support for student assessment, and the institutional uses and impacts of student assessment 
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information. The nature and results of each of these activities are addressed in the following 
sections of this report. 



1.2 Prior Research Literature 

1.2.1 Focusing the Literature 

An extensive literature review examined what is currently known about the organizational 
and administrative context for student assessment in postsecondary institutions (Peterson, 
Einarson, Trice, & Nichols, 1997). This review focused on documents published between 1985 
and 1996 and included literature from the following data bases: Education Resources Information 
Center (ERIC) system. Dissertation Abstracts International (DAI), H. W. Wilson Hies (which 
includes the Business Periodicals Index, the Humanities Index, and the Social Sciences Index), 
ABI Inform, Psycinfo, and the Social Sciences Citation Index. 

A two-phase literature search was conducted. In the initial phase, search terms related to 
different sectors of postsecondary education (higher education, postsecondary education, two-year 
colleges, and college) were cross-referenced with six search terms related to student assessment 
(student assessment, student outcomes, learning assessment, learning outcomes, outcomes 
assessment, and outcomes of education). This stage of the search yielded a total of 3,475 citations 
related to student assessment within postsecondary education institutions. 

In the second search phase, the search criteria were restricted to include only those 
documents that addressed institutional-level issues in student assessment Results from the initial 
search phase were cross-referenced with 57 search terms reflecting an array of potential external 
and institutional dimensions. This search produced 567 citations. This subset of documents was 
then evaluated and further narrowed based upon the following criteria: direct relevance to the 
research focus (document dealt specifically with institutional-level issues of student assessment in 
higher education), publication credibility (document was published in a professional journal, 
monograph, book, or report), and/or substantive content (document was empirically or 
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conceptually grounded). A total of 29 1 documents which met these criteria were included in the 
literature review for this study. 

The framework shown in Figure 1 was developed to organize the review of the literature on 
institutional support for student assessment. 

Figure 1.1 Literature Review Framework 




The framework consists of five environments: external influences on student assessment, 
institutional approach to student assessment, organizational and administrative support for student 
assessment, institutional utilization and impact of student assessment, and institutional context. 
Using this framework as a guide, the content of each of these documents was abstracted and 
analyzed. Before summarizing the results of the literature review within each environment of this 
framework, some observations about the nature of the available literature on this topic are offered. 

1.2.2 Nature of Student Asses sment Literature 

All abstracted documents were classified according to the type of publication they 
represented and the nature of the document Table 1.1 displays the types of publications 
abstracted. The majority of documents appeared as journal articles, monograph and book chapters. 
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and unpublished reports. Comparatively few documents were in the form of books and 
monographs. 

Table 1.1. Type of publications abstracted 



Unpublished Reports 


70 


Monograph and Book Chapters 


65 


Journal Articles 


64 


Conference Proceedings 


16 


Books 


10 


Dissertations 


10 


Monographs 


10 



The nature of each document was characterized using the following categories: descriptive 
(described what assessment approaches or support practices states, institutions, etc., had adopted), 
evaluative (provided a review or critique of an assessment approach or support practice), 
prescriptive (advocated/prescribed the use of an assessment approach or support practice), 
conceptual (provided a conceptual model or applied theory to the analysis of an assessment 
approach or support practice), and empirical (reported findings regarding assessment approaches 
or support practices based on systematic observation, data collection, and descriptive and/or 
relational analysis). These descriptors were not mutually exclusive, thus documents could appear 
within more than one category. The nature of the abstracted documents, as Table 1.2 reveals, is 
indicative of the recent and emerging character of the literature on student assessment The 
literature on student assessment at the institutional level is primarily descriptive or prescriptive in 
nature. Many of the documents reviewed consisted of descriptions of student assessment practices 
at single institutions or prescriptive guidelines for how institutions should support student 
assessment efforts. Little empirically-based literature and even less evaluative, conceptual or 
theoretical writing has been produced on this topic. 
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Table 1.2. Nature of Abstracted Documents 



Descriptive 


146 


Evaluative 


38 


Prescriptive 


87 


Conceptual 


27 


Empirical 


58 



Several limitations of the empirical literature must be noted. The empirical literature was 
divided about equally between those based on survey and those on case study research designs 
Survey research has primarily focused on describing the measures and approaches to student 
assessment adopted by institutions. The sampling designs of many surveys have been limitwH in 
terms of the type of postsecondary institutions included or geographical scope of the survey. Few 
studies collected information on organizational and administrative support practices or student 
assessment uses and impacts, and even fewer examined relationships among external and internal 
dimensions of assessment support. Most of the multi-case study reports reviewed were conducted 
as dissertation research and many of the single case studies lacked methodological rigor. Although 
some case study research offered a more detailed examination of the dynamics of institutional 
approaches to, and support for student assessment, this comprehensiveness was countered by the 
limited scope of organizational and administrative dime nsions considered and the limireH 
generalizability of these findings. 

The literature on student assessment is very much an emerging arena of study. As is 
evident from the results of the first and second phases of the literature search, a relatively s mall 
proportion of the documents addressed student assessment in relation to institutional-level concerns 
or topics. Few of the documents that have taken an institutional perspective have considered the 
nature of external influences on institutions’ student assessment efforts, provided systematic 
evidence on organizational and administrative support patterns, or presented little more than 





5 



anecdotal information on the institutional uses and impacts of assessment. In particular, there has 
been a paucity of research examining the relationships among these environments. 

Findings from the extant literature regarding each of the five environments of the literature 
review framework (Figure 1.1) are provided in the following sections of this report. Each section 
summarizes the literature under the specific domains identified within each of these environments. 
We also examine proposed relationships among the various environments and the nature of the 
evidence regarding these propositions. 

1 - 2 - 3 Studies of External Influences on Student Assessmen t 

The literature identified five domains of the external environment that have been discussed 
as potential influences upon institutions’ student assessment approaches, support practices, and 
uses and impacts: national efforts, state-level initiatives, regional and professional accreditation 
associations, the private sector, and professional higher education associations. 

According to many scholars, activities at the national level provided the initial impetus for 
student assessment in postsecondary education (Banta & Moffett, 1987; Ewell, 1991; Hutchings & 
Marchese, 1990; Marchese, 1987; Sims, 1992). These activities include the publication of reports 
critical of the quality of higher education (Bennett, 1984; National Institute of Education, 1984), 
development of the National Education Goals (Education Commission of the S tates [EC S ], 1991; 
Nettles, 1995), revisions to the Department of Education’s Criteria for Recognition of Accrediting 
Agencies (Sims, 1992), and linking the provision of federal funding for financial aid eligibility to 
institutions’ student assessment efforts (Banta, 1991; Cook, 1989). As a result of these latter two 
actions, all six regional accreditation agencies now require institutions to conduct student 
assessment (Cole, Nettles & Sharp, 1997). The adoption of the National Education Goals brought 
governors and state higher education leaders into the discussion of national standards and 
assessment measures for student achievement (Lenth, 1993; 1996; Nettles, 1995). The impact of 
these federal policy actions suggests that the national domain has indirectly influenced institutions’ 
decisions to engage in assessment efforts. There is limited evidence of a direct relationship 
between national activities and institutions’ assessment efforts. While some institutions have 



reported that federal funds provided significant support for their initial assessment efforts (Amarin, 
Schilling, & Schilling, 1993; Banta, 1991; Katz, 1993), data from nationally-representative 
surveys reveal that national activities have been a minor influence on institutions’ decisions to 
establish (Johnson, Prus, Andersen, & El-Khawas, 1991) or to increase (El-Khawas, 1995) their 
student assessment activity. 

In contrast to the weak and largely indirect impact of national initiatives, scholars have 
portrayed state-level actions as having a strong direct influence on institutions’ student assessment 
efforts (Aper, Cuver, & Hinkle, 1990; Ewell, 1993; Hutchings & Marchese, 1990). Since the 
mid-1980s an increasing number of states have enacted student assessment initiatives (National 
Center for Higher Education Management Systems [NCHEMS], 1996). Many institutions have 
identified state mandates among the reasons cited for initiating (Johnson et al., 1991) and 
increasing (El-Khawas, 1995) their student assessment programs and have used student 
assessment information for reports to state agencies (El-Khawas, 1990). But scholars debate 
whether state mandates have promoted institutional support for and use of student assessment or 
have mainly evoked a compliance response on the part of institutions (Aper et al., 1990; Ewell, 
1993; El-Khawas, 1995). 

States use a variety of governance structures for higher education and vary widely in the 
form that student assessment initiatives take (e.g., policy, statute, both policy and statute) (Cole, 
Nettles & Sharp, 1997). Scholars have proposed specific dimensions of state-level student 
assessment initiatives that may influence the extent to which institutions support and utilize student 
assessment. Those include whether: 1) the primary purpose of the initiative is institutional 
improvement rather than external accountability (Aper & Hinkle, 1991 ; Ewell, 1991), 2) the 
initiative is linked to other state policy levers (Ewell, 1991) or is consistent with other external 
policies or reporting requirements (Jones & Ewell, 1993; Lincoln, 1990; McGuinness, 1994), 3) 
decisions about broad guidelines for assessment and strategic and operational decisions are 
centralized at the state level or decentralized to institutions (Ewell, 1984; Jones & Ewell, 1993), 4) 
the student performance indicators and assessment instruments are selected by institutions or 
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prescribed by states (Ewell, 1987a, 1994; Jacobi, Astin, & Ayala, 1987; Ory, 1991; Terenzini, 
1989), 5) the institutions are required to report how they have utilized student assessment 
information or just provide evidence of assessment plans or the existence of student performance 
information (Ewell, 1987b, 1990), and 6) the states provide fiscal resources to support or reward 
institutions’ assessment activities (Banta, 1988; Banta & Moffett, 1987; Ewell, 1987b). 

Changes in patterns of state-level student assessment initiatives have been documented over 
the past decade and include: greater emphasis on external accountability as the primary purpose 
(Ewell, 1991, 1997; Ewell & Jones, 1993; McGuinness, 1994; NCHEMS, 1996), increasing 
linkages with other state policies or regulatory systems (NCHEMS, 1996), more frequent use of 
common performance indicators (Ewell, 1994; Gaither, 1995; Ruppert, 1994, 1995) and common 
assessment instruments (NCHEMS, 1996; Steele & Lutz, 1995), and the introduction of 
performance-based funding approaches (Cole et al., 1997; Ewell, 1991, 1997). Overall, this shift 
toward increasing regulation at the state level may be expected to increase the number of 
institutions reporting student assessment activity but decrease the level of institutional support for 
student assessment and the likelihood that assessment information is used for institutional 
improvement. 

There has been little systematic examination of the relationship between these changing 
patterns of state-level activity or the specific dimensions of state-level initiatives and institutions’ 
student assessment efforts, suppdrt practices or uses. Case study research has produced 
conflicting findings regarding the effectiveness of centralized or decentralized state approaches in 
promoting institutional support for student assessment (Aper & Hinkle, 1991; Banta, 1988; Ewell 
& Boyer, 1988). Survey research has only asked institutions to self-report whether or not a state 
requirement for student assessment exists and the extent to which stale requirements have 
influenced institutional decisions to begin or increase assessment activity (El-Khawas, 1990; 
Johnson et al., 1991; Muffo, 1992). 

Although the role of accreditation associations has received less attention in the assessment 
literature than that of state agents, accreditation agencies, particularly regional institutional 
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accrediting associations, appear to be an important domain of external influence on institutions’ 
student assessment efforts (Aper et al., 1990; Banta, 1993; Ewell, 1993). As noted previously, all 
regional accreditation associations now require member institutions to undertake and document 
some form of student assessment activity (Cole et al., 1997). However, regional association 
guidelines vary in terms of the emphasis placed on student assessment compared to other required 
indicators of institutional performance, the nature of institutional reporting requirements regarding 
student assessment efforts, and the range of student assessment-related services and activities 
provided to member institutions (Cole et al., 1997). 

Increasing numbers of institutions have reported conducting student assessment as a part of 
regional (El-Khawas, 1989, 1990, 1991, 1992, 1995) and professional (El-Khawas, 1991, 1992, 
1995) accreditation self-studies. Further, accreditation requirements was the external domain most 
frequently cited by institutions as a reason for initiating (Johnson et al., 1991; Muffo, 1992) and 
increasing (El-Khawas, 1995) their student assessment efforts. Beyond these general findings, 
evidence regarding the influence of accreditation policies and practices on institutional support for, 
and use of student assessment is scant and inconclusive (Cowart, 1990; Gentemann & Rogers, 
1987; Gill, 1993; Kalthoff & Lenning, 1991). 

Within the private sector, the business community and private foundations have been 
suggested as emergent sources of influence on institutions’ support for student assessment 
Scholars point to the inclusion of employment-related measures (e.g., student success in finding 
employment employer satisfaction with graduates) in institutions’ student assessment approaches 
as proof of the growing impact of business community interests (Banta, 1991; Ewell, 1991). Data 
from surveys of assessment approaches conducted across different types of institutions suggest 
research universities are least likely and community colleges are most likely to collect information 
on this aspect of student performance (Cowart, 1990; Johnson et al., 1991; Ory & Parker, 1989). 
Discussions of private foundation influences have centered on their provision of funding for 
institutions’ student assessment projects. Descriptions of student assessment projects reveal that a 
variety of institutions have received some financial support from private foundations for these 
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efforts (Banta, 1991 ; Banta & Moffett, 1987; Obler, Slark, & Umbdenstock, 1993). No 
systematic research was found regarding the extent or nature of business community or private 
foundation influences on institutions’ support for or use of student assessment. 

Finally, in an effort to encourage and support institutions’ student assessment activities, 
professional higher education associations have undertaken a variety of efforts that include the 
sponsorship of national conferences, publication of resource materials, and provision of consulting 
services (Banta, 1991 ; Mentkowski, 1991). No research was located regarding the extent to which 
institutions have used these services and the relationship of professional association efforts to 
institutions assessment approaches, support practices and uses of student assessment. 

Summary of External Influences. External domains must be included in any examination 
of influences on institutional support for student assessment From the perspective of institutional 
informants, accreditation requirements have been most influential in motivating institutions to 
engage in student assessment followed by state-level initiatives and, to a much lesser extent 
national efforts (El-Khawas, 1995; Johnson et al., 1991). Scholars have advanced propositions 
regarding the influence of specific dimensions of these external domains to institutional support for 
student assessment. However, the corresponding research is scant more often general than 
specific in its approach, and descriptive rather than relational. 

1-2.4 Studies of Institutional Approac hes to Student Assessment 

II 

Institutional approach to student assessment refers to the content and technical aspects of 
student assessment This domain represents the specific aspects of student performance and 
functioning an institution chooses to assess as a part of its student assessment efforts and the 
means by which it measures those aspects. The literature identified four dimensions as the basis 
for comparing institutional approaches to student assessment: content or type of student assessment 
measures, level of aggregation at which assessment occurs, timing of assessment measures, and 
type of student assessment methods employed. 

Institutions select which aspects of students’ characteristics and experiences to examine in 
their assessment approach. Astin (1991) differentiates between comparatively fixed or variable 
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student characteristics. Fixed characteristics include ascriptive circumstances such as socio- 
demographic status. Several taxonomies have been developed as means of classifying the variable 
dimensions of postsecondary students’ performance or functioning (cf. Astin, 1991; Bowen, 

1977; Lenning, Lee, Micek, & Service, 1977; Alexander & Stark, 1986; Ewell, 1984, 1987c). 
Although these classification schemes vary in terms of the specific terminology used and categories 
of variables proposed, all distinguish among aspects of students’ cognitive, affective or behavioral 
functioning. The cognitive dimension includes basic skills (reading, writing and computational 
skills), higher-order cognitive processes (critical thinking and problem solving), subject-matter 
knowledge, and general education competencies. The affective dimension includes students’ 
values, attitudes, aspirations, self-ratings, and satisfaction. The behavioral dimension includes 
observable aspects of students’ functioning such as course completion, degree attainment, hours 
spent studying, field of employment, and job performance. 

In addition to considering dimensions of students’ performance or functioning, institutions 
may also choose to examine aspects of students’ experiences within the institution as a part of their 
assessment approach (Astin, 1991; Banta et al., 1996; Erwin, 1991a; Lenning, 1991). Micek and 
Amey (1974) proposed five categories of institutional characteristics, processes and practices that 
may influence student learning and development: the instructional environment (e.g., course-taking 
patterns, teaching methods), the social environment (e.g., student-faculty contact, student 
participation in co-curricular activities, residence arrangements), the fiscal environment (e.g., type 
or amount of financial aid received, participation in work study), the organizational environment 
(e.g., faculty-student ratio, admissions policies, advising policies), and the physical environment 
(e.g., classroom space, availability of study areas, library resources). Student assessment 
approaches that examine multiple dimensions of student functioning and that include aspects of 
students’ institutional experiences are expected to contribute more to institutional decision making 
and the improvement of student performance than assessment approaches that focus narrowly on 
student attributes and performance (Astin, 1991; Ewell, 1988b; Hutchings, 1990; Johnson, 
McCormick, Pros, & Rogers, 1993). 
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The content of student assessment approaches has been examined in a number of studies. 

\ review of survey research reveals most assessment approaches have emphasized measures of 
cognitive aspects of student functioning. Of these, basic skills were most often assessed, followed 
by knowledge in the major, general education, and higher-order cognitive processes (Cowart, 

1990; Johnson et al., 1991; Ory & Parker, 1989). In comparison, behavioral variables were 
measured less often (Cowart, 1990; Gill, 1993; Patton, Dasher-Alston, Ratteray, & Kait, 1996) 
and affective variables were least likely to be measured (Cowart, 1990; Gill, 1993; Johnson et al., 
1991; Patton et al., 1996; Steele & Lutz, 1995; Steele, Malone, & Lutz, 1997). Few institutions 
collected information on students’ experiences within or perceptions of the institutional 
environment as a part of their student assessment approaches (Cowart, 1990; Gill, 1993; Ory & 
Parker, 1989; Patton et al., 1996; Steele et al., 1997). There is some evidence of differences in the 
content of student assessment approaches by institutional type (Cowart, 1990; Steele et al., 1997; 
Steele & Lutz, 1995). Compared to four-year institutions, two-year institutions were more likel y 
to assess basic skills and employment-related measures (Hexter & Lippincott, 1990) and less likely 
to assess non-cognitive measures (Kalthoff & Lenning, 1991). 

A second dimension of student assessment approaches is the level of aggregation or unit of 
analysis toward which the approach is oriented (Ewell, 1988b, 1991c; Terenzini, 1989). 
Institutions may use student assessment to examine the performance of individual students or 
student subgroups, academic courses, programs or departments, schools or colleges within the 
institution, or the institution as a whole (Alexander & Stark, 1986; Astin, 1991; Ewell, 1984, 
1987c). In general, some level of disaggregation in student assessment is deemed important to 
avoid masking differences in performance among subgroups of students (Astin, 1991; Ewell, 
1988b). Scholars have offered arguments for directing assessment approaches at various units of 
analyses (Halpem, 1987; Hlebowitsh, 1995; Loaker & Mentkowski, 1993; Ratcliff, 1995; 

Seybert, 1994). The relative effectiveness of a decision regarding unit of analysis is expected to 
depend on its congruence with an institution’s purpose for conducting assessment (Alexander & 
Stark, 1986). No research was located that examined institutional choices regarding this student 
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assessment approach dimension or the relationship of this dimension to assessment support or 
uses. 

In terms of timing of student assessment measures, institutions may collect student 
assessment data as students enter the institution, at various points during students’ enrollment, as 
or after students terminate their formal involvement with the institution, or at some combination of 
these points in time (Astin, 1991; Terenzini, 1989). Each approach to the timing of measures has 
strengths and limitations (Astin, 1991; Terenzini, 1989). Despite their associated psychometric, 
statistical and methodological complexities (Hanson, 1988; Jacobi et al., 1987; Terenzini, 1989), 
assessment approaches that incorporate multiple points of data collection are thought to have the 
potential to make a greater impact on student and institutional performance than those that collect 
data at only one point in time (Astin, 1991; Halpem, 1987; Jacobi et al., 1987; Kells, 1992). 

A synthesis of findings from research on student assessment practices undertaken across 
various types of postsecondary institutions shows data collection was most likely to occur at only 
one point in time, most often as students entered institutions and to a lesser extent at the time of 
students’ exit (Cowart, 1990; Gill, 1993; Hexter & Lippincott, 1990; Kalthoff & Lenning, 1991; 
Patton et al., 1996). There was little evidence of institutions assessing students at various points 
during their enrollment or measuring changes in students’ performance over the duration of their 
enrollment (Cowart, 1990; Gill, 1993; Hexter & Lippincott, 1990; Kalthoff & Lenning, 1991; Ory 
& Parker, 1989; Patton et al., 1996; Steele et al., 1997; von Destinon, Ganz, & Engs, 1993). 

As a final dimension of student assessment approach, institutions select specific methods to 
collect student assessment data. A basic choice concerns whether externally-developed or 
institutionally-developed methods or instruments are used (Johnson et al., 1993; Lenning, 1991; 
Ory, 1991 ; Winston & Miller, 1994). Externally-developed methods most often take the form of 
comprehensive objective examinations or inventories administered in a written or computerized 
format These have mainly been developed to measure various aspects of students’ cognitive 
functioning. To a lesser but increasing extent externally-developed instruments that measure 
students’ affective and behavioral functioning are also available. Institutionally-developed methods 



include numerous options. The literature discusses the following general categories: 
comprehensive tests or examinations (Ewell, 1987c; Fong, 1988; Johnson et al., 1993), 
performance-based measures such as projects, demonstrations, internships, simulations, or 
portfolios (Banta et al., 1996; Fong, 1988; Johnson et al., 1993; Lenning, 1988), surveys or 
interviews (Ewell, 1987c; Johnson et al., 1993; Lenning, 1988), external examiners (Fong, 1987; 
Johnson et al., 1993; Payne, Vowell, & Black, 1991), or archival records (Ewell, 1987c; Johnson 
et al., 1993; Lenning, 1988). 

There are advantages and disadvantages associated with the use of externally-developed 
and institutionally-developed assessment methods (cf. Ewell, 1984, 1987a; Jacobi et al., 1987; 
Ory, 1991; Terenzini, 1989). The effectiveness of any particular method may depend on the 
purpose of student assessment If assessment is being undertaken to meet external accountability 
requirements, externally-developed methods may be most appropriate while institutionally- 
developed methods may be a better choice if improving students’ or institutional perfo rman ce is the 
primary purpose of assessment (Ewell, 1987a; Jacobi et al., 1987; Ory, 1991). Overall, scholars 
advocate the use of multiple assessment methods to capitalize on the strengths and combat the 
deficiencies of any one method (Ewell, 1984, 1988b; Halpem, 1987; Jacobi et al., 1987; Lenning, 
1991; Ratcliff, Jones et al., 1997; Sims, 1992; Terenzini, 1989) and to permit triangulation of 
assessment results (Jacobi et al., 1987; Lenning, 1988). 

Extant research provides conflicting evidence of the student assessment methods used by 
postsecondary institutions. In two studies, institutions used externally-developed instruments 
more often than institutionally-developed methods (Johnson et al., 1991; Kalthoff & Lenning, 
1991) but the converse was found in two other surveys (Ervin, 1988; Steele & Lutz, 1995). Data 
from Campus Trends surveys show an increasing number of institutions were developing their 
own assessment methods, including the use of student portfolios (El-Khawas, 1992, 1995). 
However, in research conducted by Gill (1993) and Johnson and colleagues (1991), portfolios 
were among the least commonly-used student assessment methods. Two studies found institutions 
made greatest use of externally-developed and institutionally-developed objective tests (Smith, 
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Bradley, & Draper, 1993; von Destinon et al., 1993); according to several other studies, 
institutionally-developed methods have most often been comprised of archival data such as 
enrollment figures, course completion and course grades, and retention, graduate, and employment 
rates (Cowart, 1990; Gibson, 1992; Gill, 1993; Patton et al., 1996; Steele & Lutz, 1995). Little 
research has examined differences in student assessment methods among types of institutions 
(Steele et al., 1997). 

Summary of Institutional Approaches. Scholars have advocated the use of comprehensive 
student assessment approaches in which institutions use a variety of assessment methods to collect 
information on numerous aspects of students’ performance and experiences at multiple points in 
time. It appears most institutions have adopted relatively limited assessment approaches. 

Measures of students’ cognitive functioning have been emphasized while measures of affective and 
behavioral functioning are less common. Data have most often been collected at one point in time. 
Evidence suggests that institutions are making greater use of institutionally-developed assessment 
methods but this use can range from the mining of archival data to the development of student 
portfolio methods. The comprehensiveness of student assessment approaches undertaken by an 
institution is expected to be positively associated with the institutional uses and impacts of student 
assessment. No research was found regarding the relationship of institutional approach 
dimensions to institutional support for or uses of assessment 

1.2.5 Organizational and Administrative Support for Student Assessment 

Five domains of organizational and administrative support for student assessment have 
been discussed in the literature: institutional support strategy, leadership and governance patterns 
for student assessment, assessment management policies and practices, student assessment culture 
and climate, and evaluation of the student assessment process. 

Institutional support strategy refers to an institution’s choices about the overall purpose, 
structure, and functions of its student assessment efforts. These strategic choices represent an 
institution’s efforts to establish a fit between its external and internal environments (Peterson, 
Cameron, Mets, Jones, & Ettington, 1986). Accordingly, two general categories of institutional 



support strategy dimensions have been discussed: those reflective of external forces for student 
assessment (external assessment support strategy), and those reflective of internal forces for 
student assessment (internal assessment support strategy). 

Scholars have distinguished two dimensions of institutions’ external assessment support 
strategies that may influence assessment approaches, support practices, and uses. These are: 
whether or not an institution must respond to external mandates for student assessment (Aper et 
al., 1990; Ewell, 1991, 1993), and the timing of an institution’s assessment activities relative to the 
establishment of external assessment mandates (Ewell, 1994; Ewell & Boyer, 1988; Neal, 1995). 

Data from multi-institutional survey research support a positive relationship between the 
existence of external requirements for student assessment and the likelihood that institutions will be 
engaged in some form of student assessment activity (El-Khawas, 1990, 1995; Hexter & 
Lippincott, 1990; Johnson et al., 1991; Scott, 1991). Descriptions of student assessment efforts 
undertaken at a variety of institutions suggest that institutions whose student assessment efforts 
were initiated prior to, or concurrently with external assessment mandates have more 
comprehensive student assessment approaches and stronger internal support than institutions 
whose assessment efforts followed the imposition of external mandates (Banta, 1985, 1988; 
Hutchings & Marchese, 1990; Krueger & Heisserer, 1987). No systematic comparative research 
or relational analyses were found regarding these external strategy dimensions. 

Three dimensions of internal assessment support strategy have been proposed as important 
influences on institutions’ student assessment approaches, support practices, and uses: whether an 
institution’s assessment support strategy primarily addresses external or internal purposes 
(Braskamp, 1991; Ewell, 1987a; Hutchings & Marchese, 1990), the planning processes used for 
student assessment (Ewell, 1987c, 1988a), and the linkage between student assessment efforts and 
an institution’s academic mission (Loacker & Mentkowski, 1993; Winston & Miller, 1994). 

The dimension of purpose of student assessment efforts spans the categories of external 
and internal support strategy. While assessment strategy must address both internal and external 
purposes (Aper et al., 1990), scholars have proposed that assessment strategies that emphasize 
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internal purposes such as improving students’ and institutional performance will encourage more 
comprehensive student assessment approaches, gamer stronger internal support, and result in 
greater utilization of assessment information than strategies that emphasize external purposes such 
as fulfilling external accountability requirements (Ewell, 1987a; Hutchings & Marchese, 1990; 
Sell, 1989). Characteristics of effective student assessment planning processes have been 
suggested including developing a formal student assessment plan (Braskamp, 1991), using 
incremental planning steps such as conducting an inventory of existing student assessment activity 
(Banta et al., 1996; Payne, Vowell, & Black, 1991; Terenzini, 1989; Thomas, 1991), and 
mounting pilot projects (Ewell, 1984, 1987b, 1988a; Terenzini, 1989). Scholars have asserted the 
importance of relating an institution’s student assessment efforts to its academic mission (Lo acker 
& Mentkowski, 1993; Winston & Miller, 1994). Greater internal support for student assessment 
is expected if the mission prioritizes teaching and learning (Banta, 1993; Hutchings & Marchese, 
1990) and student assessment (Duvall, 1994) as institutional activities and clearly specifies 
intended educational outcomes (Braskamp, 1991). 

Empirical support for these propositions is in short supply. Descriptive evidence from 
1 survey (Johnson et al., 1991; Muffo, 1992) and case study research (Hyman, Beeler, & Benedict, 
1994; Suchanic, 1989/1990) suggests assessment support strategies which give equal or greater 
weight to internal assessment purposes than to external assessment purposes are associated with 
more comprehensive student assessment approaches. There is limited evidence that a growing 
number of institutions have developed formal student assessment plans (Gill, 1993; Patton et al., 
1996), that publicly-controlled institutions are more likely than private institutions to have a formal 
assessment plan, and that the number of planning steps undertaken varies by institutional type 
(Patton et al., 1996). Profiles of assessment practices reveal that some institutions have examined 
and revised their mission statements as a consequence of initiating student assessment efforts 
(Banta et al., 1996). 

Leadership and governance patterns comprise a second domain in this environment. 
Academic leadership is conceived as playing a critical role in supporting an institution’s student 
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assessment efforts (Banta & Associates, 1993; Banta et al., 1996; Braskamp, 1991; Jacobi et al., 
1987; Rossman & El-Khawas, 1987). Three dimensions of leadership support have been 
discussed: patterns of participants in providing student assessment leadership, forms of leadership 
support, and leadership styles. Strong support from the president and senior administrators is 
viewed as crucial for institutional success in student assessment efforts (Banta et al., 1996; Duvall, 
1994; Ewell, 1988a; Rossman & El-Khawas, 1987). Gaining the support of formal and informal 
leaders among an institution’s faculty and staff has also been recommended (Banta, 1993; Sell, 
1989; Young & Knight, 1993). Leadership support may take the form of co mmuni cations 
regarding the importance (Duvall, 1994; Eisenman, 1991; Peacock, 1994; Sell, 1989b) and 
purpose (Banta & Associates, 1993; Rossman & El-Khawas, 1987; Terenzini, 1989) of student 
assessment and the commitment of resources to assessment initiatives (American College Testing, 
■M 1990; Eisenman, 1991; Jones & Ewell, 1993; Miller, 1988). A participatory leadership style has 
generally been advocated as most effective for promoting internal support for student assessment 
(Banta et al., 1996; Dixon, 1994; Ewell, 1988a). 

A comparative case study of community colleges found that having faculty take on 
leadership roles for student assessment was related to the effectiveness of their student assessment 
programs (Lang, 1993). Beyond this study, evidence for the role of leadership support is based 
on descriptions of successful student assessment approaches at various campus (Banta & 
Associates, 1993; Banta et al., 1996; Knight & Lumsden, 1990). No systematic examination of 
the relationship of different forms of leadership support or leadership styles to institutional support 
for student assessment was found. 

Governance patterns refers to the administrative structures and processes used for makin g 
student assessment decisions. Discussions of administrative structures for student assessment 
consider the assignment of responsibilities for overseeing student assessment decisions to 
positions, organizational levels and functional areas within an institution (Ewell, 1984, 1987a, 
1988a, 1988b; Nichols, 1991; Sims, 1992; Terenzini, 1989; Thomas, 1991). Situating student 
assessment responsibilities in academic affairs is thought to encourage the most inte rna l support for 
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assessment (Ewell, 1984, 1987a). Student assessment decision making processes vary in the 
degree to which authority for student assessment decisions is centralized within an institution’s 
upper hierarchical levels or organizational units or decentralized across institutional levels and units 
(Banta & Associates, 1993; Banta et al., 1996; Ewell, 1984). Scholars have generally advocated 
the use of decision making processes that are decentralized (Astin, 1991; Banta et al., 1996; Ewell, 
1984; Math er, 1991) and utilize significant involvement of administrators (Miller, 1988; Rossman 
& El-Khawas, 1987; Winston & Miller, 1994), faculty (Banta & Associates, 1993; Braskamp, 
1991; Eisenman, 1991; Sell, 1989) and, to a lesser extent, students (Johnson et al., 1993; 
Somervell, 1993; Thomas, 1991). 

A few studies have examined the administrative structures used for student assessment 
Johnson and colleagues (Johnson et al., 1991) found that executive responsibility for student 
assessment was most often positioned in academic affairs, operational responsibility was more 
often given to administrators than to faculty, and less than half of respondent institutions had 
created a separate office for student assessment. In Cowart’s (1990) survey of two-year colleges, 
assigning a coordinator for student assessment was positively associated with the breadth of 
student assessment information collected and internal perceptions of the effectiveness and 
importance of student assessment. A meta-analysis of comparative case studies (Riggs & 
Worthley, 1992) revealed that the assessment expertise of project coordinators was an important 
predictor of achieving positive assessment impacts. There is some evidence that centralization of 
decision making varies across the phases of planning, implementing, evaluating, and using student 
assessment (Johnson et al., 1991; Patton et al., 1996) and with institutional size (Patton et al., 
1996). One study found faculty involvement in implementing student assessment decisions was a 
strong predictor of achieving positive outcomes from assessment efforts (Riggs & Worthley, 
1992). 

Assessment management policies and practices arc considered a powerful means through 
which institutions can support and enhance the effectiveness of their student assessment efforts 
(Ewell, 1988a; Sell, 1989b). A number of content dimensions of institutional policies have been 



identified as potential influences on student assessment and specific practices have been 
recommended within each dimension. Policies and practices related to resource allocation, 
professional development, faculty evaluation and rewards, and academic planning and review 
figure prominently in the literature. To a lesser extent, dimensions of student assessment 
information systems, communication policies, and student-related policies have been discussed. 

Administrators are urged to commit adequate fiscal, physical and staff resources to student 
assessment (Braskamp, 1991; Eisenman, 1991; Miller, 1988; Ryan, 1993; Thomas, 1991) and to 
consider linking units’ student assessment efforts to institutional resource allocation decisions 
(Ewell, 1987a, 1987b, 1987c, 1988a; Gill, 1993; Thomas, 1991). The provision of professional 
development on assessment-related topics (Banta et al., 1996; Ewell, 1988b; Gentemann, Fletcher, 
& Potter, 1994; Young & Knight, 1993) and incentives or rewards (Astin & Ayala, 1987; Ewell, 
1988b; Krueger & Heisserer, 1987) to faculty and administrators is expected to enhance their 
participation in assessment efforts. Conflicting opinions are offered as to whether institutions 
should include faculty involvement in student assessment among performance evaluation criteria 
for tenure and promotion (Ewell, 1984; Halpem, 1987; Ryan, 1993; Twomey, Lillibridge, 
Hawkins, & Reidlinger, 1995). Building formal linkages between assessment activities and 
processes for the planning and review of academic programs and departments (Chaffe-Stengel, 
1992; Ewell, 1988a, 1997), curriculum (Ewell, 1984, 1988a, 1997; Hlebowitsh, 1995), and 
student academic support services (Erwin, 1991b; Hanson, 1982) is expected to increase internal 
support for assessment and the utilization of assessment information. In addition, scholars have 
stressed the need for institutions to develop comprehensive student assessment information 
databases (Astin & Ayala, 1987; Bray & Kanter, 1996; Sell, 1989b); to establish policies and 
practices facilitating the communication of student assessment purposes, activities and results with 
a broad range of internal and external constituents (Banta et al., 1996; Ewell, 1984, 1988a; Knight 
& Lumsden, 1990; Ryan, 1993; Terenzini, 1989; Thomas, 1991); and to devise policies that 
promote the involvement of student affairs personnel (Erwin, 1991b; Hanson, 1982) and students 
(Duvall, 1994; Loacker & Mentkowski, 1993; Van Stewart, 1996) in assessment efforts. 
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There is limited descriptive evidence regarding the extent to which institutions have used 
policies and practices regarding communication (Patton et al., 1996), student assessment 
information systems (Astin & Ayala, 1987; Gill, 1993), faculty development (Steele & Lutz, 

1995), and program review (Barak & Sweeney, 1995) to support student assessment activities. 
One study found institutions’ practices regarding the intended audience for assessment reports was 
predictive of achieving positive outcomes from student assessment projects but resource allocation 
practices were not (CSUTTL, 1993). Beyond this study, evidence concerning the relationship of 
assessment management policies and practices to external influences or to institutions’ assessment 
approaches, degree of internal support and utilization of student assessment information was not 
located. 

Student assessment culture and climate constitute a fourth domain of the organizational and 
administrative environment (Banta & Associates, 1993; Banta et al., 1996; Braskamp, 1991; 

Miller, 1988). These terms have been used interchangeably in the higher education literature but 
they are conceptually distinct and may have different implications for institutions attempting to 
support student assessment efforts. Thus, their proposed relationships to student assessment and 
empirical evidence for these relationships will be considered separately for culture and climate. 

Institutional culture refers to the unique and enduring constellation of deeply embedded 
values, beliefs, and ideologies collectively held by members about their institution (Peterson, 1988; 
Peterson, Cameron, Mets, Jones, & Ettington, 1986; Peterson & Spencer, 1990). More 
specifically, student assessment culture refers to members’ perceptions of an institution’s 
purposes, values, and philosophy related to student assessment. Scholars (Banta et al. 1996; 

Jones & Ewell, 1993; Wolff & Harris, 1994) have suggested assessment-supportive cultures have 
the following characteristics: members perceive the institution values teaching and learning (Banta 
& Associates, 1993; Eisenman, 1991; Hutchings & Marchese, 1990), student assessment (Banta et 
al., 1996; Mentkowski, 1991; Ryan, 1993; Sell, 1989b), and innovation and risk-taking 
(Braskamp, 1991; Kells, 1992; Ryan, 1993), and a participatory governance style is used for 
making decisions regarding student assessment (Jacobi et al., 1987; Kells, 1992; Mentkowski, 
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1991). The literature search did not locate any research that explicitly examined student assessment 
culture. 

Climate has been defined as “current organizational patterns of important dimensions of 
organizational life, together with members’ perceptions and attitudes toward them” (Peterson, 

1988, p. 3 1). Three dimensions of climate have been distinguished. “Objective climate” refers to 
observable patterns of organizational behavior. The domains of assessment approach, assessment 
support strategy, leadership and governance patterns, and assessment management policies and 
practices discussed previously are dimensions of the objective climate for student assessment. 
“Perceived climate” refers to members’ perceptions and beliefs about how the organization does or 
should function (Peterson, 1988; Peterson et al., 1991). Asking administrators, faculty, staff or 
students how their institution assesses student performance or supports student assessment 
provides a measure of the perceived climate for student assessment “Motivational climate” refers 
to members’ feelings or attitudes about the institution, its practices, and their role wi thin it 
(Peterson, 1988; Peterson et al., 1991). Measures of members’ commitment to, involvement in, 
or satisfaction with student assessment fit the construct of motivational climate for student 
assessment. 

Scholars have suggested the following various dimensions of the organizational and 
administrative environment that may enhance the perceived and motivational climate for student 
assessment: 1) members believe that internal' improvement rather than internal or external 
accountability is the primary purpose of student assessment (Braskamp, 1991; Eisenman, 1991; 
Ewell, 1988b; Jacobi et al., 1987), 2) members believe the student assessment approach is 
congruent with the institution’s mission and values (AAHE, 1992; Braskamp, 1991; Terenzini, 
1989), 3) institutional leaders are perceived as supporting student assessment (AAHE, 1992; 
Braskamp, 1991; Ewell, 1988a; Jacobi et al., 1987), 4) a participatory governance approach is 
used to make assessment-related decisions ( Ewell, 1984, 1988b; Kells, 1992), 5) adequate 
resources are allocated for assessment efforts (AAHE, 1992; Banta et al., 1996), 6) assessment is 
integrated with processes for planning and resource allocation (Ewell, 1984, 1988a), and 7) 



incentives or rewards are provided for members who participate in assessment (Eisenman, 1991; 
Ewell, 1984; Hutchings & Marchese, 1990; Thomas, 1991). 

Campus Trends survey data reveal that a high proportion of academic administrators were 
concerned about the possible misuse of assessment information by external agencies (El-Khawas, 
1988, 1992, 1995). A comparison of findings from studies of institutions with comprehensive 
student assessment approaches (Hyman et al., 1994; Johnson et al., 1991) and those with less 
extensive assessment efforts (Muffo, 1992; Ory & Parker, 1989) suggests a negative association 
between internal concerns of this nature and the extent of internal support and involvement in 
student assessment. There is some evidence of a positive association between perceived leadership 
support for student assessment and the extensiveness of student assessment efforts undertaken by 
institutions (Jemmott, 1992/1993; Scott, 1991) and between faculty involvement in assessment 
activities and the improvement of faculty attitudes toward assessment (CSUITL, 1993). These 
measures of members’ perceptions and attitudes regarding student assessment appear to fit the 
constructs of perceived and motivational climate. The literature search located no studies that 
purported to explicitly examine the influences upon and impacts of student assessment climate. 

Scholars have characterized evaluation as one of the most important aspects of student 
assessment activity (Banta et al., 1996; Dennison & Bunda, 1989; Ewell, 1988b). Institutions 
have been urged to continually and systematically evaluate and revise their approaches to student 
assessment (AAHE, 1992; Banta et al., 1996; Loacker & Mentkowski, 1993; Sell, 1989b; Sims, 
1992; Wolff, 1992). Guidelines (Nichols, 1991; Sims, 1992; Thomas, 1991) and criteria 
(National Forum on Assessment, 1992; Ory, 1992) for evaluating student assessment programs 
are available. 

There has been little systematic research conducted regarding this domain. In one study 
(Patton et al., 1996), very few institutions had evaluated their student assessment approach. No 
studies were found that examined influences on institutions’ decisions to evaluate student 
assessment actiyity or the relationship of evaluative efforts to the utilization of student assessment 
information. 



Summary of Organizational a nd Administrative Support. The literature suggests specific 
strategy dimensions and associated institutional practices related to student assessment support 
strategy but few studies have included variables from this domain. Some descriptive evidence is 
available regarding the existence of external assessment mandates, purposes of assessment, and the 
nature of student assessment planning, but no research was found that systematic all y examined the 
relationship of specific external and internal support strategy dimensions to assessment 
approaches, other support practices, or uses of assessment information. Despite the importance 
attributed to leadership in shaping internal support for assessment, there has been virtually no 
empirical examination of its associated dimensions. Descriptive evidence is available regarding 
administrative structures and decision making processes used for student assessment but there has 
been limited analyses of the relationship of these dimensions to student assessment approaches, 
support, and uses. Assessment management policies are viewed as important by which 
institutions can support assessment efforts and a number of specific management practices have 
been advocated. The extant literature offers little descriptive evidence regarding the extent to which 
institutions have used these policies and practices and provides even less evidence of their 
relationship to assessment support or utilization. Similarly, there has been considerable scholarly 
discussion of the influence of culture and climate on institutions’ student assessment efforts, yet 
empirical support regarding influences on and the effects of culture and climate is scant Finally, 
there has been little consideration of institutions’ practices with respect to evaluating their student 
assessment approaches or the relationship of evaluation practices to other framework domains. 

Without exception, the domains of the organizational and administrative support 
environment are discussed in the literature as potentially powerful influences on the nature and 
effectiveness of institutions’ student assessment efforts. Many practices have been prescribed 
within each domain and propositions offered regarding their relationships to institutional support 
for and utilization of student assessment Evidence of the influence of these domains and their 
specific dimensions on student assessment efforts is more often based upon scholas’ or 
practitioners’ observations than on systematic research. Consequently, little is known about the 
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patterns of organizational and administrative activity that effectively support student assessment 
and promote the use of assessment information. 

1.2.6 Studies of Institutional Uses and Impacts of Student Assessment 

The higher education literature clearly contends that the assessment of student performance 
should not be undertaken as an end in itself but as a means to improve students’ and institutions’ 
performance (AAHE, 1992; Banta & Associates, 1993; Ewell, 1987b, 1988b;1997; Jacobi et al., 
1987). Two domains of student assessment information use and impacts are discussed in the 
literature: the utilization of student assessment information in academic decision making, the 
institutional impacts of student assessment 

Institutional strategic decisions and academic management policies and practices are two 
areas of institutional decision making recommended as arenas for utilizing student assessment 
information. Included among the strategic decisions that may make use of assessment information 
are academic planning decisions (Ewell, 1987a, 1987b, 1997), revising institutional mission and 
goals (Banta et al., 1996; Ewell, 1984; Jacobi et al., 1987), and institutional resource allocations 
(Ewell 1984, 1987a, 1987b, 1987c, 1988b). In general, institutions have reported using student 
assessment information most often in planning decisions (Cowart, 1990; El-Khawas, 1989b; 
Johnson et al., 1991; Steele & Lutz, 1995) and to a lesser extent, in resource allocation decisions 
(Cowart, 1990; Ory & Parker, 1989). 

Similarly, scholars have encouraged and practitioners have described the use of student 
assessment information in several areas of academic management policies and practices including 
academic program review (AAHE, 1992; Ewell, 1988a, 1997; Gentemann et al., 1994), 
professional development planning (Banta et al., 1996; Knight & Lumsden, 1990), faculty 
evaluation criteria (Ewell, 1988b; Jones & Ewell, 1993), reward structures for faculty and 
administrators (Ewell, 1984, 1988b; Thomas, 1991), and student support services (Banta, 1985; 
RiCharde, Olny, & Erwin, 1993; Williford & Moden, 1993). There is limited evidence from 
survey research that institutions have used student assessment information most often in decisions 
about program review (Barak & Sweeney, 1995) and student support services (Hyman et al., 



1994, Ory & Parker, 1989), and least often in decisions about faculty development and faculty 
rewards (Cowart, 1990; Steele & Lutz, 1995). 

Discussions of the institutional impacts from student assessment fall within two domains: 
impacts on dimensions of internal performance such as student performance, faculty behavior, 
curriculum, and student assessment culture and climate; and impacts on external indicators of 
institutional performance. 

The ultimate criterion of the effectiveness of a student assessment approach is whether it 
results in changes that improve student learning and development. Overall, the literature offers 
limited evidence of this type of impact (Banta & Associates, 1993; Banta et al., 1996). Two multi- 
institutional studies offer conflicting views from institutional respondents as to whether assessment 
efforts resulted in improved student performance (CSUTTL, 1993; Johnson et al., 1991). The 
majority of evidence is derived from the experiences of single institutions. A few institutions have 
reported increases in student achievement on standardized ex aminati ons (Bowyer, 1996; Krueger 
& Heisserer, 1987; Magruder & Young, 1996) and mean grade point averages (RiCharde et al., 
1996). More often, institutions have reported changes in indirect measures of student performance 
such as increases in student retention (Blanzy & Sucher, 1992; Walleri & Seybert, 1993) and 
student involvement in learning (Friedlander, Murrell, & MacDougall, 1993; Krueger & Heisserer, 
1 987; Williford & Moden), and more positive student attitudes regarding educational experiences 
(Krueger & Heisserer, 1987; Williford & Moden, 1993). 

Descriptions of assessment practices at a variety of institutions contend that student 
assessment efforts have stimulated changes in the teaching methods (Banta et al., 1996; Banta & 
Moffett, 1987; Lang, 1993; Friedlander et al., 1993; Walleri & Seybert, 1993; Young & Knight, 
1993) and course-embedded assessments of student learning (Katz, 1993; Loacker & Mentkowski, 
1993; Williford & Moden, 1993; Young & Knight, 1993) used by faculty. Evidence from multi- 
institutional research regarding the impact of student assessment on faculty instructional and 
assessment practices is both less available and less convincing (Cowart, 1990; CSUTTL, 1993). 
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Evidence from multi-institutional research (Cowart, 1990; CSUITL, 1993; El-Khawas, 
1989a, 1995) consistently reveals curriculum development and revision as the most common 
institutional impact of student assessment. Descriptions of single institutions’ assessment 
experiences support these findings (Banta, 1996; Katz, 1993; Knight & Lumsden, 1990; Krueger 
& Heisserer, 1987; McClain, Krueger, & Taylor, 1986; Walleri & Seybert, 1993; Young & 
Knight, 1993). 

The literature is suggestive of changes in student assessment culture and climate associated 
with engagement in student assessment. Descriptive reports from single institutions have noted 
increased emphasis on student learning as an institutional value (Williford & Moden, 1993), 
stronger institutional identification (Krueger & Heisserer, 1987), and greater collegiality of faculty- 
administrator relationships (Friedlander et al., 1993). These impacts are implicative of changes in 
student assessment culture. As a result of their assessment activities, some institutions have 
witnessed a shift in the perception of student assessment from that of a tolerated practice to an 
integrated part of the educational process (CSUTTL, 1993; Friedlander et al., 1993; Young & 
Knight, 1993). This change is reflective of the perceived climate for student assessment Reports 
of enhanced commitment to student assessment on the part of faculty (CSUTTL, 1993; Hutchings 
& Marchese, 1990) and administrators (Johnson et al., 1991) are consistent with the motivational 
climate for student assessment. However, extant research has not explicidy examined changes in 
this domain of institutional functioning. 

The association between institutional involvement in student assessment and relationships 
with the external environment has received limited consideration. The most commonly reported 
external use of student assessment information is to respond to state and accreditation reporting 
requirements (Banta et al., 1996; Banta & Moffett, 1987; Cowart, 1990; El-Khawas, 1989, 1995; 
Ory & Parker, 1989). A few institutions have attributed increases in institutional reputation 
(Young & Knight, 1993; Williford & Moden, 1993; McClain et al., 1986) and allocations of state 
funding (McClain et al., 1986) to their assessment activities and results. No systematic empirical 
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examination of the relationship between institutional student assessment efforts and external 
impacts was located. 

Summary of Uses and Impac ts of Assessment. On the basis of available research, it 
appears that student assessment information is used most often in decisions regarding academic 
planning and academic program review. Curricular development and revision is the most 
frequently reported impact attributed to student assessment. There is limited evidence regarding the 
influence of student assessment on student performance or faculty behavior, and less regarding 
associated external impacts. Most extant knowledge— about whether and how institutions have 
utilized student assessment information and how the impacts it has produced affects the 
institution — comes from participant observation in single institutional field settings (Banta & 
Associates, 1993; Banta et al., 1996). To a lesser extent, empirical studies using case study or 
survey research methods have examined this environment. Almost without exception, the evidence 
produced has been descriptive. No examples of comparative research regarding institutional 
engagement in student assessment efforts and impacts achieved were found. Furthermore, there 
has been little systematic, empirical research regarding the relationship of external influences, 
internal support practices, or assessment approaches to student assessment utilization and impact 
(CSUITL, 1993). 

1.3 Conceptual Framework 

On the basis of the literature review, we developed a conceptual framework of ins titutional 
support for student support (see Figure 1.2 next page). This framework represents how 
institutions respond to external pressures for student assessment, how they approach student 
assessment, how they organize their organizational and administrative patterns to support student 
assessment, and how student assessment information is used by and impacts ins titutions Further, 
it contends that the nature and patterns of influences on student assessment may vary with broad 
features of institutional context. 

This framework is comprised of the five environments discussed in the literature review: 
external influences on student assessment, institutional approaches to student assessment. 



Figure 1.2 Conceptual Framework of Institutional Support for Student Assessment 
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organizational and administrative support for student assessment, institutional uses and impacts of student 
assessment, and institutional context. The literature review permitted the identification of conceptual domains and 
dimensions with each environment. Table 1.3 displays the conceptual domains and dimensions in this framework 
and their definitions. 



Table 1.3 Definitions of Conceptual Framework Dimensions 
Domains and Dimensions Definitions 



External Influences 



National efforts 



State-level initiatives 



Regional accreditation 
association efforts 



Private sector (business, 
foundations) support 



• national-level activities credited with stimulating student assessment in 
postsecondary institutions (e.g., national reports on undergraduate 
education and student assessment, development of National Education 
Goals, revision of criteria for recognizing accrediting agencies, financial 
incentives or support for institutions undertaking student assessment) 

• state governance structure for higher education (consolidated governing 
board, coordinating board with regulatory authority, coordinating board 
with advisory capacity, planning agency) 

• form of student assessment initiative (statute, policy, combination, 
none) 

• specific dimensions of student assessment initiatives (purpose of 
student assessment initiative, locus of assessment initiative decision- 
making, requirements for student performance indicators and assessment 
instruments, institutional reporting requirements for student 
assessment, criteria for state evaluation of institutions* student 
assessment activities, resources provided for student assessment) 

• student assessment-related reporting requirements (evidence of 
assessment plan, assessment results, intended or actual uses of 
assessment information) 

• provision of student assessment-related services (resource materials, 
conferences, workshops, consultation) 

• inclusion of employment-related measures in student assessment 
approach 

• provision of funds for student assessment 



Professional association 
support 



• provision of student assessment-related services (resource materials, 
conferences, workshops, consultation) 



Institutional Approach to Student 
Assessment 



Extent by content 



Timing 



Assessment methods 



• extent to which institutions collect various types of student assessment 
data (e.g., cognitive, affective, behavioral) 

• whether student assessment data is collected from students at one or 
more points in time 

• methods used to collect student assessment data (e.g., quantitative or 
qualitative, developed by institution or by external sources) 
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Table 1.3 continued 



Assessment studies 

Organizational and Administrative 
Support for Student Assessment 

Institutional support strategy 



Leadership and governance 
patterns 



Assessment management 
policies and practices 



Assessment culture and climate 



Evaluation of student 
assessment process 

Institutional Context 

Institutional type 

Control 

Size 

Institutional Uses and Impacts of 
Student Assessment 

Use of assessment information 
in institutional decisions 



Institutional impacts of student 
assessment information 



• nature and number of analyses conducted and reports produced of student 

assessment data 



• institutional mission emphasis (undergraduate education, intended 
educational outcomes, student assessment) 

• purposes of student assessment (internal improvement, state or 
accreditation requirements) 

• leadership and governance activities addressing or promoting assessment 

• policies, structures and processes for planning and coordinating student 
assessment (e.g., nature of assessment plan or policy, participants in 
planning process, designation of executive and operational 
responsibility for assessment) 

• existence and extent of formally organized policies, activities and 
procedures intended to support the collection and use of student 
assessment information (e.g., resource allocation, information 
management, student involvement, professional development, faculty 
evaluation and rewards, academic planning and review) 

• institution’s purposes, values and philosophy related to student 
assessment 

• members* perceptions and attitudes concerning institution’s student 
assessment efforts and their role in these efforts 

• institutional evaluation of student assessment process 



• institutional type (associate of arts, baccalaureate, master’s, doctoral, 
research) 

• public or private control 

• institutional size (enrollment) 



• influence of assessment information in decisions concerning strategic 
decisions or academic planning 

• influence of assessment information in decisions concerning faculty 
promotion or rewards 

• impact of student assessment information on faculty behavior and 
attitudes (e.g., interest in teaching, teaching methods used) 

• impact of student assessment information on student performance (e.g., 
retention/graduation, grade performance) 

• impact of student assessment information on institution's external 
relationships (e.g., student applications, state funding, institutional 
reputation) 
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External Influences. It is apparent that demands from a variety of external constituencies 
have played an important role in initiating and shaping student assessment efforts within 
postsecondary institutions. In particular, direct influences have been exerted by state-level 
initiatives, regional accreditation associations, professional higher education associations and the 
private sector. National efforts have largely played an indirect role in influencing institutions’ 
assessment activities. 

Institutional Approach to Student Assessment. External influences and the internal 
organizational and administrative environment may shape the design and implementation of an 
institution’s approach to student assessment. Important dimensions along which student 
assessment approaches can be differentiated include: the content or type of student assessment data 
collected, the timing of student assessment measures, the methods used to collect student 
assessment data, and the level at which assessment data is aggregated for analysis. 

Organizational and Adm inistrative Support for Assessment. Five domains of 
organizational and administrative support for assessment were identified: institutional support 
strategy for assessment; leadership activities and governance patterns supporting student 
assessment; assessment management policies and practices; institutional evaluation of the student 
assessment process; and the culture and climate for student assessment. This study addresses only 
the first four of these domains of organizational and administrative support 

Institutional Context. Broad institutional characteristics such as institutional type, control 
and size are expected to moderate external influences on assessment the institutional approach to 
student assessment organizational and administrative support patterns, and institutional uses and 
impacts of assessment. 

Institutional Uses and Impacts of Student Assessment. The primary concern of this 
framework is to examine the relationship of external influences, institutions’ assessment 
approaches, and patterns of organizational and administrative support for assessment to the 
institutional uses and impacts of assessment info rmati on 
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1.4 Research Questions 



This research has been guided by the following specific questions: 

1 . What types of measures and approaches to student assessment have institutions adopted? 

2. What is the nature of external influences for student assessment in postsecondary institutions? 

3. What organizational and administrative support patterns for student assessment have 
institutions developed? 

4. How have institutions used student assessment information and what impacts has it had? 

5. How do patterns of external influences, student assessment approach, organizational and 
administrative support, and uses and impacts of student assessment vary by institutional type 
and control? 

6. How are external influences related to institutional adoption of various approaches to student 
assessment, patterns of organizational and administrative support, and uses and im par ts of 
student assessment information? 

7 . How are institutional approaches to and organizational and adminis trative support patterns for 
student assessment related to uses and impacts of student assessment information? 

8. What is the relative influence of external factors, institutional approach to student assessment, 
and patterns of organizational and administrative support for assessment on institutional use 
and impacts of student assessment information? How does this influence vary by institutional 
type? 



In chapter two, we describe the design and administration of our survey of institutional 
support for student assessment. In chapters three through seven, we present the results of 
descriptive analyses of survey data. In chapter eight, we describe the approaches used to reduce 
individual variables in our data into indices. Chapters nine through eleven present the results of 
bivariate and multivariate analyses of survey data. We summarize and discuss our survey results 
in chapter twelve. 
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2. Survey Design and Methods 

2. 1 A National Institutional Survey 

2.1.1 Nature. Purpose and Focus of the Survey 

This study, which addresses the research questions in section 1.4, involves a national 
survey of institutions of postsecondary education. This research is the first comprehensive 
national survey of how institutions approach student assessment, their patterns of organizational 
and administrative support for student assessment, and the uses and impact of those efforts. 

The survey instrument was designed as an objective, quantifiable inventory of these 
institutional approaches, support patterns, and uses and impacts of student assessment. It was 
intended to assist institutions in obtaining a clearer picture of their own efforts, to provide a 
national profile of student assessment efforts, and to analyze the research questions addressed by 
this study. 

The survey instrument and this study focus on student assessment of undergraduate 
education. They do not address assessment of graduate or continuing education. Postsecondary 
institutions are the primary unit of analysis — not individuals or academic sub-units. 

2.1.2 Survey Population 

The population for this survey is all public and private postsecondary institutions 
recognized by the U.S. Office of Education that offer undergraduate programs at the associate or 
baccalaureate degree level. This population includes institutions from all Carnegie Classifications 
(Associate of Arts through Research Universities). Specialized institutions and those not offering 
undergraduate education were excluded from the population. Neither were proprietary institutions 
included. In 1997 after eliminating the specialized institutions and those not offering associate or 
bachelors degrees, the U.S.O.E. recognized 2,524 institutions. The survey was sent to all these 
institutions — no sampling was involved. 
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2.2 The Survey Instrument and Database 



2.2.1 DesignineJhe Instrument 

This national survey is the second phase of a four phase research program ex amini ng 
organizational and administrative support for student assessment. Phase One, conducted during 
1996-97, involved an extensive review and synthesis of the literature on student assessment 
(Peterson et al, 1997). This literature review was summarized in section 1.2 and provided the 
conceptual framework discussed in section 1.3 that identified the major environments of external 
influences on, institutional approaches to, organizational and administrative support for, and 
institutional use and impacts of student assessment. 

The literature review of these environments used two approaches to identify questionnaire 
items to be included in these environments. First, the dynamics, policies, or practices mentioned 
or reported in the literature were included as dimensions or items in the questionnaire. Second, we 
identified instruments used in other surveys of institutions on student assessment. Ite ms in these 
instruments were reviewed for possible inclusion. Based on these sources, a preliminary 
instrument was designed to examine institutional activities, policies and practices in the 
environments of the conceptual framework. 

2.2.2 Pilot Studies 

The preliminary instrument was pilot tested with chief academic administrators in four 
different types of institutions: community college, liberal arts college, regional public university, 
and research university. In half the cases, the respondents completed the questionnaire and were 
then interviewed by one of our researchers. In the other half, the researcher sat in the room with 
the respondent and urged them to discuss their reactions as they completed the questionnaire. 

These pilot tests led to substantial revisions of some areas of the questionnaire, the addition and 
elimination of some items, and the clarification of others. 
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2.2.3 Content of the Instrument and Database 



The instrument for this national institutional survey is included as Appendix I. It is 
organized in five major sections reflecting the conceptual framework and includes 244 items. 
Table 2.1 portrays the primary sections (conceptual domains) and subsections of the questionnaire 
and identifies the questionnaire items related to each. 



Table 2.1. Dimensions of Institutional Support for Student Assessment 



Dimension of Institutional Support 


Survey Questions 


External Influences on Student Assessment 




National efforts 


mCla-b 


State-level initiatives 


mAi-5, mcic, mc2c 


Regional accreditation associations 


mBl-3,mC2b 


Private sector support 


mcid 


Professional association support 
Institutional Approach to Student Assessment 


mC2a, d 


Content 


IA1-14 


Timing 


IA1-14 


Methods 


IB1-10, IC1-9, ID 1-4 


Assessment studies 


IE1-10, IF1-6 


Organizational and Administrative Support for 
Student Assessment 




Institutional support strategy 


IIAl-2,IIBl-7 


Leadership and governance patterns 


nci-7,nDi-6,nEi-9 


Assessment management policies and practices 


IVA1-4, Bl-4, Cl-5, Dl-6, El-4, Fl-7, 
Gl-7, Hl-4 


Culture and climate for student assessment « 


Not included in this survey 


Evaluation of student assessment process 
Institutional Uses and Impacts of Student 
Assessment 


IIF1-2 


Decision making 


VA1-12 


Internal impacts 


VB1-8 


External impacts 


VB9-15 



Several dimensions reflected in the conceptual framework and included in the data b a se for 
this study were drawn from two sources other than the survey instrument The p rimar y 
Institutional Context Variables (Institutional Type, Control, and other characteristics) are from the 
IPEDS database. Relevant state indicators identifying state characteristics, policies and practices 
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related to the states’ role in student assessment were drawn from NCPI Project 5.1’s survey of 
state policies for student assessment (Cole, Nettles, & Sharp, 1997). 

2.3 Conducting the Survey 

2.3.1 Identifying Respondents 

Since the study was designed as an objective, quantifiable national population survey of 
institutional activities, a complete mailing list of all U.S.O.E. postsecondary institutions was 
obtained. The survey was personally addressed to the chief academic officer at each institution. 
While the chief academic officer was the primary point of contact, it was not assumed that this 
individual would complete the instrument Consequently, the cover letter encouraged them to have 
the questionnaire completed by the person or group which had the most comprehensive 
understanding of the institution’s student assessment activities. (See Appendix II • B). 

2.3.2 The Survey Process 

The actual survey process included five steps. First a preliminary letter which informed 
the chief academic officer of the nature and importance of this national study and the impending 
receipt of the questionnaire was sent two weeks in advance (See Appendix II - A). Second, the 
survey instrument was sent to the chief academic officer with a cover letter indicating its intended 
use as an institutional self study inventory as well as directions for completing and returning it (See 
Appendix II B). Third, a reminder postcard was mailed about a week after mailing the instrument 
(See Appendix II - C). Fourth, about one month following the mailing of the questionnaire, all 
non-responding institutions received a personal phone call from a member of our research team 
encouraging them to respond and offering to answer questions. A second mailing of the survey 
was sent targeting non-responding institutions as well as those contacted by phone who had 
requested another survey (See Appendix II - D). Throughout the entire survey process members 
of the research team were available by phone or e-mail to respond to any questions. The response 
rate prior to the phone follow up was 19%. An additional 36% responded after the phone calls. 
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Finally , about two months after the initial mailing, a thank you letter was sent to all responding 
institutions (See Appendix II - E). 



2.4 Data Coding and Entry 

As each institutional response was received, they were recorded, reviewed for accuracy of 
response, and checked to assure the IPEDS institutional identifier number was still Att a ched (to 
allow for merging institutional characteristics from the IPEDS database). 

The questionnaires were sent to a commercial firm for computerized data entry. All entries 
were double verified. A complete computer disk of all institutional responses was transferred to 
the research team when data entry was concluded. A random check of several questionnaires 
revealed no errors in data entry. 

The survey data were then merged with selected institutional characteristics from the IPEDS 
database and with selected state level student assessment dimensions for each institution’s state to 
form a comprehensive database for the study. A set of derived indices for each institution which 
was created during a data-reduction phase (see Section 2.5.2) would later be added to the database. 

2.5 Analysis Plan 



2.5.1 Item Review 

Data analysis included several steps. First, frequency distributions, means and standard 
deviations of all questionnaire items for all responding institutions were reviewed to identify any 
inconsistencies among similar items or items for which there was little or no variation (i.e. could 
not be used in later relational analyses). 

2.5.2 Descriptive Profiles bv Conceptual Domain and Institutional Type and Control 
Research questions one through four (section 1.1) ask what approaches to student 

assessment institutions have adopted, the nature of external influences for student assessment, 
\yhat patterns of organizational and administrative support for student assessment institutions have 
developed, and what the institutional uses and impacts of student assessment have been. These 
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will be addressed by examining the response profiles to the questionnaire items in the domains of 
each of these sections. Research question five asks how these student assessment domains vary 
by institutional type and control. These were analyzed using analyses of variance (or chi squares if 
appropriate) to examine differences on these institutional dimensions. Similarly, research question 
number six asks how external groups (primarily institutional accreditation region and state 
approaches to student assessment) influence the institutional domains of student assessment 
These were also analyzed using analysis of variance (or chi square analysis as appropriate). 
Research question seven asks about the relationships among institutional domains of student 
assessment. These relationships were examined using correlations. 

2.5.3 Data Reduction 

The items in the questionnaire were designed to identify institutional patterns related to the 
domains in the conceptual framework. In order to ascertain whether such patterns among 
institutional policies, procedures, and activities existed, a data reduction was attempted. The intent 
was both to examine the patterns of related institutional policies, practices, and activities and to 
reduce the number of variables to be included in the relational analysis. 

Because the response categories in different sections or subsections of the questionnaire 
varied, it was not possible to include all items in the entire questionnaire at once. Rather individual 
sections (or subsections) of the questionnaire with items related to a common dimension were 
factor analyzed. In some instances because of the categorical nature of the item responses, we 
created an additive index. 

Each factor analysis was rotated using the oblique option. Items were included in the factor 
index that emerged if they met the following three criteria; a) were weighted most heavily on that 
factor, b) had a factor loading exceeding .40, and c) were conceptually similar in content to the 
other items in the factor. Individual items which had substantial variance but did not load heavily 
on any factor were retained as single item variables. 

Two scores were created for each index. One was based on the mean score of the items in 
the index and was to be used for descriptive summaries since the index scale would then be 



analogous to that of the item response scale in the questionnaire. The other was a normalized score 
(0 - 1) to be used in subsequent statistical regressions. 

Table 2.2 lists the resulting indices by name, how they were derived (factor analysis, 
separate item, or additive index of items not subjected to factor analysis), the alpha coefficient of 
each index, and the questionnaire items. The actual results of factor analysis are included in 
Appendix IV. 

2.5.4 Relational Analysis 

Research question number eight asks how the external influences, institutional approach 
and oiganizational and administrative support patterns affect the degree to which student 
assessment data are used and/or have positive institutional impacts. This question was analyzed 
using stepwise regression. Separate models— one for all responding institutions and then separate 
regressions by institutional type — were conducted and compared. For these relational analyses, 
we mainly used the indices derived in the data reduction phase of the analysis since the item level 
of analysis would involve too many predictor variables. 

2.6 Survey Responses 

As noted in the discussion of the survey population (section 2.1.2), the survey instrument 
was sent to 2,524 postsecondary institutions who offer undergraduate associate of arts and 
baccalaureate degrees. After all phases of follow up, we received 1,393 usable responses for an 
overall response rate of 55.1%. This rate is quite high for a national survey in which institutions 
are the unit of analysis. 

Table 2.2 Summary of Derived Variables 

Derived Variable Variable Type of Alpha Survey 

Name Variable Items 

External Influences on Stu dent Assessment no derived variables 
Institutional Approach to Student Assessment 
Extent by Content 



academic intentions 


extentl 


item 




IA1 


postcollege assessment 


posted 


factor 


.83 


IA11.12, 14 


cognitive assessment 


cognit 


factor 


.71 


IA3-6 



0 




52 



40 



Table 2.2 continued 
Extent by Content 



affective assessment 


affect 


factor 


.68 


IA7-9 


academic progress 


extent 10 


item 




IA10 


social roles 


extent 13 


item 




IA13 


comprehensiveness of data collection 


extenttt 


additive index 




IA1-10 

(extentl-14) 


Timing of data collection 


timingtt 


additive index 




IA1-10 
(time 1. 1-9.3) 


Student assessment instruments 


number of instruments 


instrtt 


additive index 




IB 1-10 


Other student assessment methods 


student-centered methods 


studmeth 


factor 


.61 


IC1-4 


external methods 


extmeth 


factor 


.63 


IC8-9 


transcript analysis 


othmethS 


item 




ICS 


external examination 


othmeth6 


item 




IC6 


interviews with withdrawing students 


othmeth7 


item 




IC7 


Student assessment studies 


curricular experience studies 


studcur 


factor 


.69 


IE1-3, 8-9 


co-cunicular experience studies 


studcoc 


factor 


.70 


IE4-7 


number of studies 


studies 


additive index 




IE1-9 


Student performance profiles or reports 


number of reports 


reports 


additive index 




IF1-5 


Organizational and Administrative Support for 
Student Assessment 

Institutional Support Strategy 


mission emphasis 


missemph 


additive index 




ILAla-c 


internal purposes 


intpurp 


factor 


.79 


IIB3-6 


accreditation purposes 


purpose 1 


item 




IIB1 


state purposes 


purpose2 


item 




IIB2 


other purposes 


purpose7 


item 




HB7 


Leadership and Governance Patterns 


administrative and governance 


govemin 


additive index 




HC1-7 


activities 


administrative and faculty support 
Leadership and Governance Patterns 


admmspt 


additive index 




nm-5 


breadth of assessment planning group 


grouptot 


additive index 




EE3 


number approving changes 


appro vtot 


additive index 




HE5 


Assessment Management Policies and 
Practices 


resource allocation practices 


resalloc 


additive index 




IVA1-4 


budget decisions 


budgfact 


additive index 




IVA3-4 


computer support 


infosyst 


additive index 




IVB2-4 


access to information 


accessin 


additive index 




IVC1-4 



rest copy available 




Table 2.2 continued 



Assessment Management Policies and 
Practices 



distribution of reports 


infodist 


additive index 




IVD1-4 


student involvement 


studinv 


factor 


.69 


IVE1.3-4 


student incentives 


ive2 


item 




IVE2 


professional development 


profdev 


factor 


.77 


IVF2-5 


faculty training required 


ivfl 


item 




IVF1 


student affairs 


staffirs 


factor 


.84 


IVF6-7 


faculty evaluation 


faceval 


factor 


.77 


IVG1-5 


hiring process 


ivg6 


item 




IVG6 


encourage faculty 


ivg7 


item 




IVG7 


academic planning and review 


planrev 


factor 


.84 


IVH1-4 


Culture and Climate for Student 


no derived variables 






Assessment 

Evaluation of Student Assessment Process 


conducted evaluation 


evaluate 


dichotomous 




IIFla-b 


Institutional Uses and Impacts of Student 


Assessment 

Institutional Decision Making 
academic decisions 


intdec 


factor 


.83 


VA1-5, 8-12 


faculty decisions 


facdec 


factor 


.79 


VA6-7 


Institutional Impacts 


faculty impacts 


teachimp 


factor 


.79 


VB1-4 


student impacts 


studimp 


factor 


.82 


VB5-8 


external impacts 


extimp 


factor 


.82 


VB9-15 



Table 2.3 displays the response rates by institutional type and control and by accrediting 
region. The response rates by institutional type vary from a low of 44% for Baccalaureate I 
institutions to a high of 76% for Research II institutions. The public institu tion response rate of 
62% was higher than that for private institutions of 53%. The response rate by accreditation region 
ranged from a low of 40% in the Western region to a high of 62% in the North Central region. A 
table of institutional response rates by states is shown in Appendix m. 
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Table 2.3 Survey Response by Institutional Type, Control and 
Accrediting Region 



Classification 


Number of Surveys 
Sent Received 


Response 

Rate(%) 


Institutional Type 








Research I 


86 


52 


60 


Research II 


37 


28 


76 


Doctoral I 


48 


27 


56 


Doctoral II 


58 


38 


66 


Masters’ I 


429 


263 


61 


Masters’ II 


89 


52 


58 


Baccalaureate I 


164 


72 


44 


Baccalaureate II 


432 


244 


56 


Associate of Arts 


1022 


548 


54 


Institutional Control 








Public 


1439 


885 


62 


Private 


951 


508 


53 


Unclassified* 


134 


69 


51 


Accrediting Region 








Middle States 


403 


191 


47 


New England 


186 


87 


47 


Northwest 


140 


81 


58 


North Central 


847 


528 


62 


Southern 


746 


423 


57 


Western 


206 


83 


40 


Total 


2524 


1393 


55 



■Carnegie classification was missing for 134 institutions; institutional control 
was missing for 23 institutions within this subset 
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3. Institutional Approaches to Student Assessment 
This chapter examines the institutional approaches to student assessment (research question 
one) and how those vary by institutional type and control (research question five). Specifically, 
the survey section on institutional approaches addressed the content or type of student assessment 
measures that institutions used, the timing of those assessments, the source of s tandardized 
instruments, the qualitative methods used, the special sub-populations of students studied, the 
types of assessment studies done, and the student performance reports and profiles that were 
prepared. This information provides the first comprehensive national picture of institutional 
approaches to student assessment. In the sections that follow, we profile the overall pattern for all 
institutions and then examine the variation by institutional type and by public or private control. 
Institutional type is examined by collapsing the nine Carnegie types (see Table 2.3) into five degree 
level groups — associate, baccalaureate, master’s, doctoral, and research institutions 

3.1 Content of Student Assessment: Type and Extent. 

Based on the literature review and the examination of institutional practices, ten types of 
student assessment measures were identified for currently enrolled students. Four additional types 
were identified for former students. The survey asked institutions to identify the extent to which 
they employed each type for assessing undergraduate students using the following scale: 1 = not 
collected, 2 = collected for some students, 3 = collected for many students, 4 = collected for all 
students. Table 3.1 displays the pattern of responses for all the responding institutions and the 
means and standard deviation for each type. 

3.1.1 T ype and Extent of Student Assessment - Currently Enrolled Students 

The respondents reported that the three most commonly used types of student assessment 
measures collected for all students were student academic progress (69.6%), basic college- 
readiness skills (60.4%) and student academic intentions or expectations (53.5%). When the 
category “collected for many students” is combined with “collected for all students,” these content 
types were used, respectively, by 82.4%, 86.4%, and 77.7% of the institutions. These three types 



Table 3.1 Extent of Student Assessment by Type of Student Assessment Data for All 



Type of Student Assessment Data 
Collected 






Extent of Data Collection (%)* 
N= 1393 






J&2L.CynsiQihLEii)r<?lle<iJStv«ienls; 


i 


2 


3 


4 


Missing 


Mean 


SD 


1. Student academic intentions or 
expectations 


9.1 


10.7 


24.4 


53.5 


2.3 


3.25 


.98 


2. Basic college-readiness skills 
(reading, writing, mathematics) 


4.7 


7.7 


26.0 


60.4 


1.3 


3.44 


.83 


3. Higber-oider skills (critical thinking, 
problem solving) 


39.3 


24.5 


17.4 


16.0 


2.7 


2.10 


1.11 


4. General education competencies 


28.5 


17.0 


21.8 


30.4 


2.3 


2.55 


1.21 


5. Competence in major field of study 


19.4 


25.8 


25.6 


25.8 


3.4 


2.60 


1.08 


6. Vocational or professional skills 


31.6 


31.5 


24.0 


9.0 


3.9 


2.11 


.97 


7. Personal growth and affective 

development (values, attitudes, social 
development) 


34.3 


29.3 


20.7 


12.8 


2.9 


2.12 


1.04 


8. Student experiences and involvement 
with institution 


16.3 


27.4 


35.6 


17.9 


2.7 


2.57 


.97 


9. Student satisfaction with institution 


3.9 


21.3 


46.4 


25.6 


2.8 


2.96 


.80 


10. Student academic progress (retention, 
graduation rates) 

For Former Students: 


.7 


2.9 


12.8 


69.6 


13.9 


3.76 


.55 


1 1. Vocational or professional outcomes 
(career goals, job attainment or 
performance) 


7.2 


28.9 


46.9 


15.3 


1.8 


2.72 


.81 


12. Further education (transfer, degree 
attainment, graduate study) 


6.7 


31.4 


45.9 


14.4 


1.6 


2.69 


.80 


13. Civic or social roles (political, social 
or community involvement) 


46.1 


28.6 


17.9 


4.2 


3.2 


1.8 


.89 


14. Satisfaction and experiences with 
institution after leaving 


8.7 


33.5 


41.8 


14.5 


1.5 


2.63 


.84 


•l=not collected; 2 = collected for some stud 


lents; 3 = 


collected for many students; 4 = collected for all students 





of student assessment data reflect a considerable attention to entry measures (intentions and basic 
skills) and objective, easily quantifiable measures of progress. 



Four types of student assessment measures were in the mid range of those reported as 
collected for many or aU students by over 50% of the institutions. When the collected for all and 
for many students are combined, the percentage frequencies of those four were: student 
satisfaction with the institution (72.0%), student experiences and involvement (53.5%), general 
education competencies (52.2%) and competence in the major field (51.4%). These categories, no 
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doubt, reflect the increased attention paid to understanding and responding to students over the past 
decade and to curricular- or program-based assessment activities. 

The three types of student assessment data most often reported as not collected were higher 
order skills ( 39.3 % do not collect), personal growth and affective development (34.3%) and 
vocational or professional skills (31.6%). When combined with the category “collected only for 
some students,” these figures increase respectively to 63.9%, 63.7%, and 63.1%. The content of 
these three types of student assessment data suggests considerably less attention is given to 
cognitive, affective, and vocational measures. In one sense, these assessment aspects are the most 
difficult to measure and probably most difficult on which to obtain faculty agreement regarding 
specific measures. 

The overall pattern suggests a substantial amount of attention paid to collecting various 
types of student assessment data. This pattern, as we shall see later, varies to some de gree by 
institutional type and control and, no doubt, reflects the considerable attention paid to these issues 
by state agencies and accrediting associations. 

3.1.2 Type and Extent of Student Assessment - Former Students 

Table 3.1 also provides some perspective on the collection of student assessment data on 
former students. For three of these types of measures, the most common response was “collected 
for many students” (all over 40%). ..When collected for many and for all students are combined, 
more than 50% of all institutions reported collecting student assessment data on: vocational or 
professional outcomes (62.7%), further education (60.3%), and satisfaction with the institution 
after leaving (56.3%). The student outcome data which was least often collected was students’ 
civic or social roles (46% not collected and 28% collected for some students). 

Once again, these figures suggest a substantial amount of interest in student assessment 
data on former students. However, this approach is mainly focused on students’ careers and 
perceptions of the institution— measures that are fairly objective and easy to ascertain. 
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3.1.3 Extent of Student Assessment bv Institutional Type - Current Students 



Table 3.2 presents an analysis of the differences among five institutional types (collapsing 
the Carnegie typology by degree level) on the extent to which various types of student assessment 



Table 3.2 Extent of Student Assessment by Institutional Type 



Type of Student Assessment 
Data Collected 


Extent of Data Collection* 


A A 
(N=545) 


Bacc 

(N=313) 


Master’s 

(N=311) 


Doctoral 

(N=64) 


Research 

(N=80) 


F 


For Currently Enrolled Students: 


Mean 


Mean 


Mean 


Mean 


Mean 




1 . Student academic intentions or expectations 


3.38 


3.33 


2.99 


3.05 


3.15 


9.41** 




(.90) 


(1-10) 


(1.03) 


(.97) 


(.95) 




2. Basic college-readiness skills 


3.56 


3.35 


3.40 


3.27 


3.08 


8.50** 




(-59) 


(.97) 


(.87) 


(1.01) 


(1.11) 




3. Higher-order skills 


1.88 


2.41 


2.25 


2.05 


1.92 


13.67** 




(1.02) 


d-20) 


(1.11) 


(1.01) 


(1.07) 




4. General education competencies 


2.41 


2.76 


2.61 


2.53 


2.32 


5.25** 




(1-18) 


(122) 


(1.18) 


(1.21) 


d-23) 




5. Competence in major field of study 


2.30 


2.92 


2.86 


2.70 


2.38 


23.97** 




(1.07) 


(1.08) 


(.98) 


(1.01) 


(1.02) 




6. Vocational or professional skills 


2.25 


1.97 


2.00 


2.10 


1.86 


6.80** 




(1.00) 


(.97) 


(.89) 


(.87) 


(.80) 




7. Personal growth and affective development 


1.77 


2.51 


2.29 


2.27 


2.57 


36.18** 




(.91) 


(1.09) 


(.98) 


(.98) 


(.99) 




8. Student experiences and involvement with 


2.35 


2.79 


2.68 


2.59 


2.78 


13.55** 


institution 


(.94) 


(1.01) 


(.93) 


(.85) 


(.86) 




9. Student satisfaction with institution 


2.86 


3.13 


2.97 


2.78 


2.88 


6.48** 




(.75) 


(.86) 


(.79) 


(.83) 


(.79) 




10. Student academic progress 


3.58 


3.87 


3.86 


3.83 


3.97 


22.10** 




(.68) 


(.45) 


(.41) 


(.38) 


(.16) 




For Former Students: 














11. Vocational or professional outcomes 


2.73 


2.74 


2.75 


2.59 


2.44 


3.06* 




(.83) 


.(•78) 


(.78) 


(.81) 


(.75) 




12. Further education 


2.71 


2.74 


2.71 


2.55 


2.42 


3.24* 




(.80) 


(-78) 


(.77) 


(.85) 


(.73) 




13. Civic or social roles 


1.37 


2.26 


2.08 


2.05 


1.86 


72.77** 




(.67) 


(.91) 


(.87) 


(.92) 


(.86) 




14. Satisfaction and experiences with institution 


2.57 


2.63 


2.75 


2.63 


2.47 


3.21* 


after leaving 


(.87) 


(.84) 


(.77) 


(.75) 


(.78) 





* l=not collected; 2=collected for some students; 3=collected for many students; 4=collected for all students 

* p < .05; **/><. 01 

Note: Differences across group means were estimated using one-way ANOVA. 
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data were collected. The mean scores of the extent to which each type of student assessment 
measure was collected are reported in the Table 3.2 (1 = not collected, 2 = collected for some 
students, 3 = collected for many students, 4 = collected for all students). ANOVAs were used to 
identify statistically significant differences among the institutional types on each type of student 
assessment data. 

Not surprisingly, there were significant differences among the institutional types at the .01 
level for all ten types of student assessment measures for currently enrolled students. Some 
general patterns deserve comment While there is a statistical difference, it should be noted that 
student academic progress measures were emphasized at all five types of institutions with mean 
scores ranging from 3.58 to 3.97, reflecting the fact that these data are collected for most or ah 
students at all five institutional types. 

Turning to the institutional types, associate of arts institutions ranked highest among the 
five institutional types in their emphasis on three types of measures: student acaderaic intentions 
(3.38), basic college-readiness skills (3.56), and vocational or professional skills (2.25). They 
were least likely to collect student assessment data on higher order skills (1.88), competence in the 
major field (2.30), personal growth and affective development (1.77), and student experiences and 
involvement (2.35). The higher emphasis on entry level performance data and on vocational or 
professional outcomes is not surprising given associate of arts institutions’ traditional role as an 
open door college with substantial occupational and vocational emphasis. The limited attention to 
a c ad emic competence measures may reflect the difficulty of dealing with these issues for large 
numbers of part-time and/or less than degree-seeking students. Their lesser interest in patterns of 
student experience and involvement is somewhat surprising given the student-orientation of man y 
associate of arts institutions. 

Baccalaureate institutions, not surprisingly, were highest among the institutional types in 
emphasizing seven types of measures that included both performance measures such as higher 
order skills (2.41), general education competencies (2.76), competence in major field of study 
(2.92), and personal growth and development (2.51 - close to research universities’ 2.57); and 
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perceptions of the environment, such as student experiences and involvement (2.79) and student 
satisfaction with the institution (3.13). Baccalaureate institutions’ greater focus on student 
assessment is further reflected in their not scoring the lowest among the institutional types in the 
extent of data collected on any of the types of student assessment measures. This pattern clearly 
reflects the image of baccalaureate institutions as being more focused on their students’ personal 
and intellectual development. 

Master’s and doctoral-level institutions were not among the highest institutional type in 
collecting data on any type of student assessment measure for current students. However, they 
also tended not to be the lowest - except master’s institutions on student academic intentions (2.99) 
and doctoral institutions on student satisfaction with the institutions (2.78). This pattern seemingly 
reflects the ambiguous role of these institutions in trying to balance both an undergraduate and a 
graduate emphasis. 

Perhaps most surprising, research universities ranked highest among the institutional types 
on their collection of two types of student assessment data: personal and affective development 
(2.57) and student academic progress (3.97). They ranked lowest in their emphasis on basic 
college-readiness skills (3.08), general education competencies (2.32), and vocational and 
professional skills (1.86). The emphasis on personal development may reflect an attempt by these 
institutions to respond to the criticism of their lack of attention to undergraduate education. Their 
lower emphasis on basic skills may reflect their selective nature, and on vocational or professional 
skills, their more academic orientation or graduate school emphasis. 

3.1.4 Extent of Student Assessment bv Institutional Type - Former Students 

The comparison among institutional types on assessment of former students is also 
portrayed in Table 3.2. The differences among these four types of measures was not as 
pronounced as it was for current students. Only civic or social roles showed differences at the .01 
level of significance among the institutional types (the other three had differences at the .05 level). 

Three institutional types were very similar in placing the greatest emphasis on collecting 
student assessment data on vocational or professional outcomes (associate of arts - 2.73, 



baccalaureate - 2.74, and master’s - 2.75). Research universities were the lowest on this measure 
(2.44). Those same three institutional types also gave greatest emphasis to assessing further 
education (associate of arts - 2.71, baccalaureate - 2.74, and master’s - 2.71). Again research 
universities gave the least emphasis to this measure (2.42). These patterns seem to reflect the 
interest that the less-than-doctoral-level institutions give to their students’ post college vocational 
and educational patterns. 

Not surprisingly, baccalaureate institutions placed greatest emphasis on assessing their 
former students’ civic and social roles (2.26) and associate of arts colleges gave it the least (1.37). 
However, as noted, the emphasis is low for all types of institutions. Satisfaction with the 
institution after leaving was emphasized most by the master’s level institutions (2.75) and least by 
the research universities (2.47). 

While attempts to collect student assessment data from former students do not present as 
much contrast among institutional types, the patterns of difference do reflect the difference in 
institutional missions among them. 

3.1.5 Extent of Student Assessment bv Institutional Control 

Because of the differing oversight of public institutions by state agencies and their interest 
in student assessment, comparisons of the extent to which public and private institutions collected 
various types of student assessment data were also made. The ANOVAs comparing the public and 
private practices are presented in Table 3.3 

Both public and private institutions emphasized to a considerable degree the 
collection of data on student academic intentions and do not differ significantly (3.22 and 331, 
respectively). However, they differed significantly on the other nine types of student assessment 
measures for currently enrolled students. Public institutions gave greater emphasis only to basic 
college-readiness skills (3.54) and vocational or professional skills (2.15). 

Although actual differences were not always large, private institutions gave greater 
emphasis to collecting data on higher order skills (2.34), general education competencies (2.71), 
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Table 3.3 Extent of Student Assessment by Institutional Control 



Type of Student Assessment 
Data Collected 


Extent of Data Collection* 


Public 

Control 

(N=873) 


Private 

Control 

(N=502) 




For Currently Enrolled Students: 


Mean 


Mean 


t 


1. Student academic intentions or 


3.22 


3.31 


- 1.79 


expectations 


(.97) 


(1.00) 




2. Basic college-readiness skills 


3.54 


3.27 


5.35** 




(.68) 


(1.01) 




3. Higher-order skills 


1.97 


2.34 


-5.68** 




(1.03) 


(1-20) 




4. General education competencies 


2.46 


2.71 


-3.68** 




(1.18) 


(1-23) 




5. Competence in major field of study 


2.45 


2.86 


-6.84** 




(1.05) 


(1.09) 




6. Vocational or professional skills 


2.15 


2.03 


2.09* 




(.95) 


(1.00) 




7. Personal growth and affective 


1.87 


2.56 


-11.68** 


development 


(.92) 


(1.10) 




8. Student experiences and involvement 


2.40 


2.86 


-8.43** 


with institution 


(.93) 


(.98) 




9. Student satisfaction with institution 


2.87 


3.13 


-5.69** 




(.75) 


(.85) 




10. Student academic progress 


3.70 


3.86 


-533** 




(.59) 


(.46) 




ForJFormerSiudents: 








11. Vocational or professional outcomes 


2.71 


2.72 


-.18 




(.80) 


(.83) 




12. Further education 


2.66 


2.75 


-2.06* 




(.80) 


(.80) 




13. Civic or social roles 


1.55 


2.61 


-14.17** 




(.78) 


(.91) 




14. Satisfaction and experiences with 


2.61 


2.66 


-.96 


institution after leaving 


(.83) 


(.85) 





• l=not collected; 2=collected for some students; 3=collected for many students; 4=collected for all students 

* p < .05; ** p < .01 

Note: Means were compared using t test for independent samples. 

competence in the major field (2.86), personal growth and affective development (2.56), student 
experiences and involvement (2.86), student satisfaction with the institution (3.13), and student 
academic progress (3.86). 
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With respect to collecting student information on former students, there were fewer 
statistically significant differences. There was no difference between public and private institutions 
in data collection that emphasizes vocational or professional outcomes or satisfaction with the 
institution after leaving. Private institutions gave slightly greater attention to their former students’ 
patterns of further education (2.75 at the .05 level) and considerably more attention to their civic 
and social roles (2.61 at the .01 level). 

These overall patterns suggest considerable difference in emphasis by public and private 
institutions on the type and extent of student assessment information collected. The direction of 
differences, however, are not surprising given the nature of public institutions. 

Summary . This review of the extent to which institutions use various types of student 
assessment measures suggests that institutions have begun to engage in a substantial am ount of 
student assessment on a variety of types of measures. There is a greater emphasis on assessing 
current rather than former students. There are statistically significant differences among 
institutions on all ten types of student performance measures used for current students and all four 
types used for former students. These institutional type differences, to a degree, reflect differences 
of institutional mission. They also suggest a higher level of emphasis overall on student 
assessment in baccalaureate institutions, a relatively low emphasis in research universities, a mixed 
or intermediate emphasis in master’s and doctoral institutions, and a high level of interest in 
associate of arts institutions in entry level and vocational or occupational measures. Public-private 
differences are also extensive and statistically significant on eleven of the fourteen types of student 
assessment measures. Despite state level interest in student assessment, the fact that privates report 
using nine of the eleven types of student assessment measures more often seems to reflect a greater 
development of student assessment activity in the private sector. 

3.2 Timing of Student Assessment 

A significant issue in the collection of student assessment information on currently enrolled 
students is when to collect that information. More importantly, student assessment experts 
encourage institutions to collect the same information at different points in time in order to examine 



the amount of change in students’ performance and the factors influencing it to see if the 
institutional experience is providing educational “value added” (Astin, 1991). Survey respondents 
were asked to indicate the timing of the collection of different types of student assessment 
information. They indicated whether it was: not collected, collected at entiy, collected while 
enrolled, or collected at exit. Respondents could check multiple responses for each type of 
assessment data collected. Table 3.4 displays the frequency of responses to this question and a 

column indicating how many times a respondent indicated the data were collected at two points in 
time. 



Table 3.4 Timing of Student Assessment by Type of Student Assessment Data for All 
Respondents 



Type of Student Assessment Data 
Collected 


Timing of Data Collection (% of institutions) 
(N = 1393) 


For Currently Enrolled Students: 


Not 

collected 


Collected at 
entry 


Collected 

while 

enrolled 


Collected at 
exit 


Collected 

twice* 


Missing 


1 . Student academic intentions or 
expectations 


9.1 


81.0 


31.8 


17.0 


25.0 


2.4 


2. Basic college-readiness skills 
(reading, writing, mathematics) 


4.7 


90.2 


19.9 


7.1 


16.3 


1.8 


3. Higher-order skills (critical 
thinking, problem solving) 


39.3 


18.3 


38.3 


57.9 


13.0 


2.8 


4. General education competencies 


28.5 


22.1 


44.1 


24.9 


15.9 


2.4 


5. Competence in major field of study 


19.4 


7.1 


47.5 


49.6 


22.7 


3.6 


6. Vocational or professional skills 


31.6 


6.7 


42.9 


38.1 


20.6 


4.5 


7. Personal growth and affective 
development (values, attitudes, 
social development) 


34.3 


20.3 


42.3 


25.9 


15.2 


2.7 


8. Student experiences and 
involvement with institution 


16.3 


5.6 


56.9 


43.4 


20.3 


4.3 


9. Student satisfaction with 
institution 


3.9 


6.0 


65.0 


59.2 


31.2 


4.4 



•collected at entry and while enrolled, at entry and exit, or while enrolled and at exit 



Student assessment data collected most often at entry were student academic intentions 
(81.0%) and basic college-readiness skills (90.2%). Those most likely to be collected on exit were 
higher order skills (57.9%) and competence in the major field (49.6% - although 47.5% collected it 
while enrolled). Measures reported as most often collected while enrolled were: general education 
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competencies (44.1%), vocational or professional skills (42.9%), personal growth and affective 
development (42.3%), student experiences and involvement (56.9%), and student satisfaction with 
the institution (65.0%). 

Perhaps most revealing is the percentage of institutions indicating they collected a particular 
type of student information twice. The range of all ten types of data that were reported as collected 
twice was from 13.0% (higher order skills) to 31.2% (student satisfaction with the institution). 
Other types of data likely to be collected twice included: student academic intentions (25.0%), 
competence in the major field (22.7%), vocational or professional skills (20.6%), and student 
experience and involvement with the institution (20.3%). It seems apparent, although the figures 
are still low, that many institutions are beginning to collect some types of student as ses sm en t 
measures more than once. 

3.3 Source of Student Assessment Instruments 

A significant issue for institutions embarking on student assessment is whether to develop 
their own instrument, rely on one provided by the state (sometimes required) or purchase a 
commercially-available instrument Respondents were asked to indicate which, if any, they used 
for the various types of student assessment measures. The responses are arrayed in Table 3.5. 

The first nine are the same as those used for currently enrolled students in Table 3. 1 and the tenth 
is the alumni satisfaction measure identified for former students in Table 3.1. The other types of 
student assessment measures were not compatible with measurement by an instrument (although 
they might be defined by a standardized index). 

3.3.1 Source of Student Assessment Instruments for all Institutions 

The high percentage of institutions reporting that they do not use any instrument for 
measuring higher-order skills (42.2%), personal growth and affective development (37.5%), 
vocational or professional skills (33.5%), and general education competencies (31.3%) is reflective 
of the fact that these student assessment measures were of the type most often reported as not being 
used (see Table 3.1). Among institutions using instruments for measuring the various types of 



Table 3.5 Source of Student Assessment Instruments by Type of Student Assessment Data for 
All Respondents 





% Institutions Using Instruments from Various Sources* 
(N=1393) 


Type of Student Assessment Data 


Not 

used 


Institutionally 

developed 


State 


Commercially 

available 


Missing 


1 . Student academic intentions or 
expectations 


20.5 


51.0 


4.2 


31.7 


2.9 


2. Basic college-readiness skills 
(reading, writing, mathematics) 


6.3 


37.6 


10.8 


67.1 


.9 


3. Higher-order skills (critical 
thinking, problem solving) 


42.2 


29.1 


2.2 


32.0 


2.7 


4. General education competencies 


31.8 


40.1 


5.2 


32.8 


2.2 


5. Competence in major field of study 


19.4 


64.3 


12.4 


39.3 


1.9 


6. Vocational or professional skills 


33.5 


42.6 


14.4 


24.7 


3.9 


7. Personal growth and affective 
development (values, attitudes, 
social development) 


37.5 


39.2 


2.3 


29.6 


2.9 


8. Student experiences and 
involvement with institution 


20.6 


60.0 


3.7 


24.2 


3.5 


9. Student satisfaction with 
institution 


3.8 


72.9 


8.2 


34.7 


1.4 


10. Alumni satisfaction and 
experiences 


9.6 


77.7 


7.8 


15.0 


1.9 



‘Institutions could select more than one source of instrument for each content area 



student performance, it is clear that they place greatest reliance on institutionally-developed 
instruments. For eight of the ten measures of student assessment, institutions relied most heavily 
on institutionally-developed instruments. Over 50% of institutional respondents reported using an 
institutionally-developed instrument for measuring alumni satisfaction (77.7%), current student 
satisfaction (72.9%), student experiences (60.0%), and student academic intentions and 
expectation (5 1 .0%). Use of commercial instruments was most common for measuring basic 
college-readiness skills (67.1%) and higher-order thinking skills (32.2%). There was, however, a 
reliance on commercial instruments by at least 30% of the institutions for six of the ten measures. 
Use of state provided instruments was quite limited on all ten measures. Also, since the response 
reported for each type of student assessment measure exceeded 100% of the responding 
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institutions (they could check more than one), it is apparent that some institutions used instruments 
from more than one source. 

3.3.2 Source of Instruments bv Institutional Type 

Table 3.6 shows the distribution of sources of the various instruments used by institutional 
type. There was a statistically significant difference among the institutional types regarding the 
non-use of a measure for seven of the ten measures of student assessment The percentage 
differences, however, tend to reflect differences in institutional mission. For example, associate of 
arts institutions differed substantially from other types of institutions and were less likely to use 
instruments to measure personal growth and development (53.5% do not use such instruments), 
higher-order thinking skills (51.3% - about the same as research universities), general education 
competencies (45.5%), competence in the major field (27.3%), and vocational or professional 
skills (44.6%). 

The reliance on institutionally-developed instruments is reflected in the fact that, despite 
some differences, over 70% of each institutional type relied on this approach for student 
satisfaction and alumni satisfaction; over 50% of each institutional type did so for competence in 
the major field and for student experiences and involvement with the institution. 

A comparison of the use of state-provided instruments among the various institutional types 
was statistically significant for six of the ten types of student assessment measures but the .. 
percentage differences were small due to the limited use of this source of instruments (see Table 
3.5). These differences were more dramatic in the comparison of public and private institutions in 
the next section (3.3.3). 

The greatest difference in sources of instruments used by differing institutional types was 
in the use of commercial instruments. There were statistically significant differences for eight of 
the ten types of instruments. The only types of student assessment measures without such 
stat is tical differences were in the use of commercial instruments for measuring general education 
competencies and for vocational or professional skills. Among those measures where there was a 
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Table 3.6 Source of Student Assessment Instruments Used by Type of Data and Institutional 
Type 



Source Used by Institutional Type (% of Institutions)' 



Type of Student Assessment Data and 
Source of Instrument 


Assoc of 
Arts 

(N=548) 


Bacc- 

alaureate 

(N=316) 


Master's 

(N=315) 


Doctoral 

(N=65) 


Research 

(N=80) 


Chi-square 


1. Student academic intentions or 
expectations 














Not used 


17.9 


21.0 


23.3 


28.3 


20.3 


6.1 


Institutionally developed 


60.1 


46.0 


46.0 


45.0 


53.2 


24.3** 


State provided 


7.6 


.3 


3.0 


3.3 


2.5 


28.5** 


Commercially available 


23.8 


41.7 


38.7 


45.0 


44.3 


43.0** 


Missing 


2.0 


2.2 


4.8 


7.7 


1.3 




2. Basic college-readiness skills 














Not used 


1.1 


11.5 


4.8 


12.9 


22.8 


81.1** 


Institutionally developed 


20.7 


49.8 


53.0 


58.1 


46.8 


129.6** 


State provided 


13.9 


3.2 


11.8 


16.1 


11.4 


26.5** 


Commercially available 


84.8 


57.8 


58.8 


38.7 


45.6 


139.3** 


Missing 


.5 


.9 


•6 


4.6 


1.3 




3. Higher-order skills 














Not used 


51.1 


39.0 


34.2 


35.4 


51.3 


29.6** 


Institutionally developed 


25.9 


29.9 


32.9 


44.6 


28.9 


12.1* 


State provided 


2.1 


1.9 


1.6 


3.1 


2.6 


.8 


Commercially available 


26.9 


37.3 


42.3 


36.9 


27.6 


24.8** 


Missing 


2.9 


2.5 


2.5 


— 


5.0 




4. General education competencies 














Not used 


34.8 


27.5 


31.3 


33.8 


45.5 


10.8* 


Institutionally developed 


38.3 


45.0 ' 


41.9 


46.2 


35.1 


5.5 


State provided 


5.3 


3.6 


5.2 


6.2 


6.5 


2.0 


Commercially available 


33.3 


36.6 


34.5 


27.7 


22.1 


7.0 


Missing 


2.9 


2.2 


1.6 


— 


3.8 




5. Competence in major field of study 














Not used 


29.9 


11.2 


8.4 


14.1 


27.3 


79.0** 


Institutionally developed 


55.2 


74.1 


76.0 


76.6 


62.3 


54.4** 


State provided 


14.4 


9.3 


15.9 


12.5 


2.6 


14.6** 


Commercially available 


25.9 


50.5 


56.8 


43.8 


35.1 


95.3** 


Missing 


2.2 


.9 


2.2 


1.5 


3.8 




6. Vocational or professional skills 














Not used 


28.8 


43.0 


36.5 


37.1 


44.6 


20.8** 


Institutionally developed 


50.4 


33.4 


43.2 


46.8 


41.9 


22.9** 


State provided 


14.0 


14.9 


18.9 


14.5 


5.4 


9.4 
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Table 3.6 continued 
Vocational or professional skills 



Commercially available 


24.7 


23.2 


29.2 


32.3 


25.7 


4.6 


Missing 


2.6 


4.4 


4.4 


4.6 


7.5 




7. Personal growth and affective 
development 














Not used 


53.5 


26.7 


28.6 


26.2 


21.1 


95.4** 


Institutionally developed 


29.0 


47.6 


45.7 


56.9 


56.6 


54.8** 


State provided 


2.4 


1.6 


2.6 


3.1 


2.6 


1.0 


Commercially available 


21.3 


41.0 


38.5 


35.4 


34.2 


46.3** 


Missing 


2.4 


2.8 


3.5 


— 


5.0 




8. Student experiences and involvement 
with institution 














Not used 


29.3 


15.6 


12.9 


18.8 


10.5 


45.0** 


Institutionally developed 


57.0 


65.8 


66.0 


65.6 


71.1 


12.4* 


State provided 


5.1 


1.3 


4.3 


7.8 


3.9 


9.9* 


Commercially available 


17.1 


35.2 


30.7 


26.6 


30.3 


39.3** 


Missing 


4.0 


2.8 


3.8 


1.5 


5.0 




9. Student satisfaction with institution 














Not used 


3.9 


5.8 


2.9 


6.2 





7.7 


Institutionally developed 


75.0 


71.7 


70.6 


73.8 


112 


2.9 


State provided 


11.5 


2.9 


8.7 


10.8 


8.9 


19.0** 


Commercially available 


27.6 


43.4 


43.9 


40.0 


32.9 


33.0** 


Missing 


1.5 


1.6 


1.6 


— 


1.3 




10. Alumni satisfaction and experiences 














Not used 


15.5 


7.1 


4.2 


3.2 


6.4 


38.1** 


Institutionally developed 


72.9 


81.4 


85.8 


84.1 


84.6 


24.4** 


State provided 


12.1 


1.3 


8.7 


6.3 


9.0 


31.1** 


Commercially available 


9.3 


22.5 


17.7 


28.6 


14.1 


36.2** 


Missing 


2.4 


1.6 


1.6 


3.1 


2.5 





‘Institutions could select more than one source of instrument for each content area. 
< .05; < .01 



statistically significant difference, the associate of arts institutions relied more than other 
institutional types on commercial instruments for one measure - basic college-readiness drills 
(84.8%); baccalaureate institutions relied most on two measures - personal growth and 
development (41.0%) and student experiences and involvement with the institutions (35.2%); 
master’s institutions relied most on three measures - higher-order thinking skills (42.3%), 
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competencies in the major field (56.8%), and student satisfaction with the institution (43.9%); and 
doctoral institutions relied most on two measures - student academic intentions and expectations 
(45.0%) and alumni satisfaction (28.6%). Research universities did not rely more than other 
institutional types on commercial instruments for any of the ten types of student assessment 
measures. 

3.3.3 Source of Instruments bv Institutional Control 

Table 3.7 presents the sources of student assessment instruments used by public and 
private institutions. These comparisons are particularly interesting in light of the extensive state- 
level interests in student assessment and, more recently, in institutional performance indicators, 
which usually include student assessment indices. 

In the response indicating instruments were not used, there were statistically significant 
difference on eight of the ten measures of student assessment. The two measures on which public 
and private institutions do not differ were student academic intentions and general education 
competencies. Public institutions were more likely to not use instruments for higher-order skills 
(45.8% did not use), competence in the major field (22.4%), personal growth and affective 
development (46.0%), student experiences with the institution (25.7%), and alumni satisfaction 
(1 1 .4%). One might note these are areas not usually high on the list of outcomes stressed by state- 
level political interests. Private institutions on the other hand were less likely to use instruments to 
assess basic college-readiness skills (12.3%), vocational or professional skills (41.7%), and 
student satisfaction (5.4%). 

In the area of institutionally developed instruments public and private institutions differed 
significantly on seven of the ten student assessment measures. The three areas with no diff erences 
included higher-order thinking skills, general education competence, and student satisfaction. The 
two areas in which public institutions used institutionally-developed instruments more were student 
academic intentions (55.3%) and vocational or professional skills (48.5%). Private institutions 
were more likely to develop their own instruments in five areas: basic college-readiness skills 
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Table 3.7 Source of Student Assessment Instruments Used by Type of Data and 
Institutional Control 



Type of Student Performance Data 
and Source of Instrument 


Source of Instrument Used by Institutional 
Control (% of Institutions)* 


Public 

(N=885) 


Private 

(N=508) 


Chi- 

Square 


1 . Student academic intentions or expectations 








Not used 


21.3 


20.9 


.02 


Institutionally developed 


55.3 


47.8 


7.2** 


State provided 


6.7 


.2 


32.1** 


Commercially available 


28.0 


40.7 


22.7** 


Missing 


2.8 


3.1 




2. Basic college-readiness skills 








Not used 


3.0 


12.3 


47.0** 


Institutionally developed 


30.0 


51.9 


65.3** 


State provided 


15.8 


2.2 


61.5** 


Commercially available 


74.5 


55.9 


50.7** 


Missing 


.8 


1.0 




3. Higher-order skills 








Not used 


45.8 


39.1 


5.7* 


Institutionally developed 


29.2 


31.2 


.6 


State provided 


2.7 


1.4 


2.3 


Commercially available 


30.9 


36.5 


4.5* 


Missing 


2.6 


3.0 




4. General education competencies 








Not used 


33.5 


30.8 


1.1 


Institutionally developed 


39.3 


43.9 


2.8 


State provided t 


6.6 


3.0 


7.8** 


Commercially available 


33.6 


33.4 


.01 


Missing 


1.9 


2.8 




5. Competence in major field of study 








Not used 


22.4 


15.2 


10.1** 


Institutionally developed 


63.2 


69.5 


5.6* 


State provided 


14.3 


9.8 


5.7* 


Commercially available 


36.5 


46.3 


12.6** 


Missing 


1.9 


1.8 




6. Vocational or professional skills 








Not used 


31.1 


41.7 


15.2** 


Institutionally developed 


48.5 


36.9 


17.0** 


State provided 


15.4 


14.4 


.2 


Commercially available 


27.4 


22.7 


3.5 


Missing 


2.9 


5.5 




7. Personal growth and affective development 








Not used 


46.0 


25.6 


55.1** 


Institutionally developed 


35.3 


49.2 


24.9** 


State provided 


3.4 


.6 


10.3** 


Commercially available 


24.5 


40.9 


39.3** 


Missing 


2.8 


3.1 
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Table 3.7 continued 








8. Student experiences and involvement with 








institution 








Not used 


25.7 


13.8 


26.5** 


Institutionally developed 


58.8 


68.2 


11.7** 


State provided 


5.8 


.6 


22.3** 


Commercially available 


20.2 


33.5 


29.2** 


Missing 


3.8 


3.0 




9. Student satisfaction with institution 








Not used 


3.0 


5.4 


5.0* 


Institutionally developed 


74.9 


72.2 


1.2 


State provided 


12.6 


.8 


58.1** 


Commercially available 


30.8 


43.0 


20.8** 


Missing 


1.2 


1.6 




10. Alumni satisfaction and experiences 








Not used 


11.4 


7.0 


6.9** 


Institutionally developed 


77.1 


83.1 


7.1** 


State provided 


12.4 


.2 


64.6** 


Commercially available 


13.2 


18.9 


7.7** 


Missing 


1.9 


2.0 





•Institutions could select more than one source of instrument for each content area. 
*p < .05; **p < .01 



(51.9%), competence in the major field (69.5%), personal growth and affective development 
(49.2%), student experiences and involvement with the institution (68.2%), and alumni 
satisfaction (83.1%). 

In terms of state-provided instruments, there were eight types of student assessment 
measures on which there were statistically significant differences. The only two types with no 
difference between public and private institutions were: higher-order thinking skills and general 
education competence - areas in which states have evidently not played an active role. On the eight 
types of measures with significant differences, not surprisingly, public institutions were more 
likely to use state-provided instruments. The type of state-provided instruments that public 
institutions were most likely to use were: basic college-readiness skills (15.8%), competence in 
the major field (14.3%), student satisfaction with the institution (12.6%), and alumni satisfaction 
(12.4%). But even these percentages were quite low suggesting the limited reliance on state- 
provided instruments. 
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Public and private institutions differed significantly on commercially-available instruments 
for eight of the ten types of student assessment measures. The two on which they did not differ 
were general education competence and vocational or professional skills. On the eight types of 
measures with statistical differences, public institutions relied on commercial instruments more 
than private institutions only for basic college-readiness skills instruments (74.5% vs. 55.9%). In 
the other seven types of measures, private institutions used commercial instruments more for 
student academic intentions (40.7%), higher-order thinking skills (36.5%), competence in the 
major field (46.3%), personal growth and affective development (40.9%), student experiences and 
involvement (33.5%), student satisfaction with the institution (43.0%), and alumni satisfaction 
(18.9%). 

Summary- By and large, institutions that engage in student assessment still tend to rely on 
institutionally-developed instruments. Institutions do rely to a moderate but lesser degree on 
commercially-available instruments (except in the case of basic college-readiness sldOs which 
draws heavily on commercial instruments). There are substantial differences among differing 
types of institutions and in the public-private control patterns. These differences often reflect the 
nature of institutional mission and the influence of state agencies in the public sector. 

3.4 Alternative Student Assessment Methods 

In recent years, there has been considerable interest in non-traditional methods of assessing 
students. Many of these alternative methods tend to be more innovative and/or qualitative — relying 
less on objective or quantitative measurement, surveys, standardized instruments, and the like. 
Nine such methods were identified in the literature. Institutions and respondents were asked to 
indicate the extent to which these methods were used at their institutions (1 = not used, 2 = used in 
some units, 3 = used in most units, 4 = used in all units). Table 3.8 presents the frequency with 
which institutions reported using these nine methods. 
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Table 3.8 Extent of Use of Other Student Assessment Methods for All Respondents 



Extent of Use* by All Institutions (% of Institutions) 
(N=1393) 





Other Student Assessment Methods 


i 


2 


3 


4 


Missing 


Mean 


SD 


1. 


Observations of student performance 


8.2 


62.7 


21.2 


6.1 


1.8 


2.26 


.69 


2. 


Student portfolios or comprehensive 
projects 


6.5 


79.0 


10.1 


3.4 


1.0 


2.10 


.54 


3. 


Student performance in capstone 
courses 


18.2 


54.6 


17.4 


7.5 


2.4 


2.15 


.81 


4. 


Student interviews or focus groups 


23.8 


67.6 


5.6 


1.4 


1.7 


1.84 


.57 


5. 


Transcript analysis 


35.5 


30.5 


10.6 


20.4 


3.1 


2.16 


1.14 


6. 


External examination of students 


8.9 


80.8 


6.7 


2.1 


1.4 


2.02 


.49 


7. 


Surveys or interviews with 
withdrawing students 


16.8 


46.2 


14.7 


20.5 


1.7 


2.40 


1.00 


8. 


Alumni interviews or focus groups 


30.0 


54.1 


8.1 


6.0 


1.8 


1.90 


.79 


9. 


Employer interviews or focus groups 


27.4 


59.7 


7.5 


3.7 


1.7 


1.87 


.70 



*l=not used; 2 = used in some units; 3 = used in most units; 4 = used in all units 



3.4.1 Extent of Use of Alternative Methods 

A careful examination of the table suggests institutions used alternative methods only in a 
limited fashion. The most frequent response on all nine methods was collected for some students. 
As indicated by their mean scores, the methods most often used were: surveys or interviews with 
withdrawing students (2.40), observations of student performance (2.26), transcript analysis 
(2.16), and student performance in capstone courses (2.15). But even these mean scores were 
only slightly above 2 (used in some units) on the response scale. Only two of the methods were 
used in all units by more than 20% of the institutions: surveys or interviews with withdrawing 
students (20.5%) and transcript analysis (20.4%). No other method was reported as used in all 
units by more than 7.5% of the institutions. 

The four least often used methods as indicated by these mean scores were: student 
interviews or focus groups ( 1 .34), employer interviews or focus groups ( 1 .87), alumni interviews 
or focus groups (1.90), and external examinations of students (2.02). Three of these four methods 
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were reported as not used by over 25% of the responding institutions: transcript analysis (35.5%), 
alumni interviews or focus groups (30.0%), and employer interviews or focus groups (27.4%). 

3.4.2 Extent of Use of Alternative Methods Bv Institutional Type 

Table 3.9 profiles the mean scores of institutional use of these other methods of student 
assessment by institutional type. The only method on which there was no significant difference 
among the five institutional types was transcript analysis— one of the four methods most frequently 
used. The other eight methods all showed statistically significant differences by institutional type. 

Associate of arts institutions reported the most frequent use of only one method: employer 
interviews or focus groups (1.98) — still low given the associate of arts institutions’ strong 
occupational emphasis and regional focus. They used four methods least frequently among the 
five institutional types: student portfolios or comprehensive projects (1.95), al umni interviews 
(1.80), student performance in capstone courses (1.78), and student interviews or focus groups 
( 1 .65). While some of these comparisons are not surprising, the low use of interviews or focus 
groups with students is unexpected. 

Baccalaureate institutions were most likely to use alternative methods of student 
assessment They reported the highest level of use compared to the other types of institutions on 
four of the eight alternative methods: surveys or interviews with withdrawing students (2.78), 
student performance in capstone courses (2.50), observation of student performance (2.34), and 
student portfolios or comprehensive projects (2.29). They were least likely among the institutional 
types to use employer interviews or focus groups (1.66). These methods seem to reflect their 
focus on individual students and on retention. 

Master’s and doctoral institutions tended to be neither highest nor lowest among the 
institutional types in their use of alternative methods. Master’s institutions were highest in using 
alumni interviews and focus groups (2.03) and doctoral institutions were highest on two alternative 
methods: student interviews and focus groups (2.06) and external examination of students (2.18). 
Master’s and doctoral institutions were not lowest in using any of the methods, reflecting their 
middle of the pack approach to student assessment 



Table 3.9 Extent of Use of Other Student Assessment Methods by Institutional Type 



Extent of Use by Institutional Type (% of Institutions) 1 



Other Student Assessment Methods 


Assoc of Arts Baccalaureate. 
(N=539) (N=315) 


Master's 

(N=314) 


Doctoral 

(N=65) 


Research 

(N=78) 


F 




Mean 


Mean 


Mean 


Mean 


Mean 




1. Observations of student 


2.22 


2.34 


2.24 


2.26 


2.00 


4.40** 


performance 


(.72) 


(.70) 


(.59) 


(.62) 


(.53) 




2. Student portfolios or 


1.95 


2.29 


2.18 


2.25 


2.04 


24.77** 


comprehensive projects 


(.46) 


(.65) 


(.50) 


(.56) 


(.38) 




3 . Student performance i n 


1.78 


2.50 


2.41 


2.47 


2.11 


62.75** 


capstone courses 


(.71) 


(.84) 


(.71) 


(.82) 


(-53) 




4. Student interviews or focus 


1.65 


1.96 


1.98 


2.06 


1.92 


30.04** 


groups 


(.54) 


(.64) 


(.47) 


(.43) 


(.42) 




5 . Transcript analysis 


2.19 


2.24 


2.13 


1.94 


1.19 


2.11 




(1.15) 


(1.18) 


(1.11) 


(1.04) 


(.94) 




6. External examination of 


2.01 


1.98 


2.08 


2.18 


1.88 


4.99** 


students 


(.43) 


(.56) 


(.46) 


(.56) 


(.53) 




7. Surveys or interviews with 


2.26 


2.78 


2.35 


2.31 


2.08 


17.25** 


withdrawing students 


(.99) 


(1.03) 


(.95) 


(.96) 


(.70) 




8. Alumni interviews or focus 


1.80 


1.95 


2.03 


1.95 


1.96 


4.73** 


groups 


(.81) 


(.78) 


(.76) 


(.72) 


(.80) 




9. Employer interviews or focus 


1.98 


1.66 


1.87 


1.86 


1.82 


10.88** 


groups 


(.75) 


(.65) 


(.58) 


(.66) 


(.66) 





* l=not used; 2=used in some units; 3=used in most units; 4=used in all units 
*p< .05; **p< .01 

Note: Standard deviations are in parentheses. Differences across group means were estimated using one-way 
ANOVA. 

Research universities, like associate of arts institutions, did not use alternative student 
assessment methods extensively. They were not the highest user among the institutional types of 
any alternative student assessment method but they were lowest on three methods: surveys or 
interviews with withdrawing students (2.00), observations of student performance (2.00), and 
external examination of students (1.88). Despite their fairly high use of traditional methods of 
student assessment (Table 3.2), they seem not to engage in alternative or less traditional methods. 



3.4.3 Extent of Use of Alternative Methods bv Institutional Control 

Table 3.10 compares the mean scores of extent of use for each of the nine methods between 
institutions of public and private control. Seven of the nine methods showed statistically 
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significant differences. The two methods for which there was no difference were transcript 
analysis and alumni interviews or focus groups. 

Table 3.10 Extent of Use of Other Student Assessment Methods by Institutional 
Control 



Other Student Assessment Methods 


Extent of Use by Control* 
(% of Institutions) 


Public 

(N=875) 


Private 

(N=504) 


t 


1 . Observations of student performance 


2.21 


2.34 


-3.39** 




(.69) 


(.70) 




2. Student portfolios or comprehensive 


2.00 


2.28 


-8.54 


projects 


(.46) 


(.63) 




3. Student performance in capstone courses 


1.97 


2.46 


-10.93** 




(.72) 


(.85) 




4. Student interviews or focus groups 


1.77 


1.97 


-5.93** 




(.53) 


(.62) 




5. Transcript analysis 


2.12 


2.24 


-1.78 




(1.10) 


(1.19) 




6. External examination of students 


2.06 


1.95 


3.77** 




(.44) 


(.57) 




7. Surveys or interviews with withdrawing 


2.21 


2.72 


-9.22** 


students 


(.93) 


(1.04) 




8. Alumni interviews or focus groups 


1.88 


1.93 


-1.22 




(.80) 


(.77) 




9. Employer interviews or focus groups 


1.97 


1.71 


6.79** 




(.69) 


(.68) 





‘ l=not used; 2=used in some units; 3=used in most units; 4= used in all units. 

*p< .05; ** p < .01 

Note: Standard deviations are in parentheses. Group means were compared using t test for independent samples. 

Public institutions used external examinations of students (2.06) and employer interviews 
or focus groups (1.97) more frequently than private institutions. In the case of external exams, 
this may reflect attempts to provide institutional evidence of student performance or program 
quality for state program reviews. 

Private institutions, on the other hand, used five other methods more than public 
institutions: surveys or interviews with withdrawing students (2.72), student performance in 
capstone courses (2.46), observations of student performance (2.34), student portfolios or 
comprehensive projects (2.28), and student interviews or focus groups (1.97). Not only did they 
use these methods more than their public counterparts, but the mean scores suggest they used them 
more frequently than the two methods on which public institutions predominate. 
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Summary. While the more qualitative alternative methods are used less frequently than the 
instruments identified in section 3.3, most institutions report using them for some students. There 
are significant differences among institutional types on eight of these nine methods. These 
differences suggest that the highest level of use of alternative methods is among baccalaureate 
institutions and that the lowest level of use is among research universities and associate of arts 
institutions. The public-private comparison indicates that private institutions are more likely to be 
engaged in the use of these other methods. This finding may suggest that state pressures for 
student assessment drive public institutions toward the more objective, quantifiable methods and 
instruments. 



3.5 Assessing Special Student Populations 

As postsecondary institutions have increased their enrollment of non-traditional (part-time, 
older) students, many have pointed out the need for special student assessment approaches to 
reflect their unique characteristics, learning styles, needs, and life situations. In the survey, 
institutions were asked whether they used different assessment methods for some of these special 
student populations. Table 3. 1 1 displays the percentage of institutions who reported using 
different assessment approaches or methods for such groups and compares them by institutional 
type and control. 

For all respondents it is clear that most institutions did not use a different assessment 
approach for special student populations. While 21.5% reported using a special approach for 
distance education students, very few adopted special approaches for adult (9.6%), part-time 
(4.9%), or minority students (2.2%). Despite the lack of any special assessment approach, 
institutions did give considerably more attention to profiling and reporting on special student 
populations or subgroups (see Table 3.13). 

There was little difference in studies of special populations by institutional type. The only 
statistical difference was on studies of adult students where baccalaureate institutions and master’s 
institutions reported using special approaches to a greater degree than the other institutional types 
(17.3% and 16.1%, respectively). Interestingly, associate of arts institutions gave least attention to 
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Table 3.11 Use of Different Student Assessment Methods for Special Student Populations by 
Institutional Type and Control 



Institutions (%) Using Different Student Assessment Methods for Special Student Populations 











Institutional Type 
(N=1305) 






Institutional Control 
(N=1366) 


Student Population 


All 

N=1366 


Assoc, of 
Arts 
N=545 


Bacca- 

laureate 

N=306 


Master’s Doctoral Research 
N=311 N=64 N=79 


F 


Public 

N=874 


Private 

N=492 


t 


i. 


Adult students 


9.6 


2.8 


17.3 


16.1 


3.1 


8.1 


68.1** 


3.4 


20.6 


106.0** 


2. 


Part-time students 


4.9 


3.0 


7.3 


4.8 


3.2 


6.7 


9.4 


3.7 


7.2 


8.4** 


3. 


Minority students 


2.2 


1.8 


1.0 


3.2 


3.1 


3.8 


5.3 


2.5 


1.6 


1.2 


4. 


Distance 

education students 


21.5 


20.9 


21.8 


25.1 


15.8 


26.7 


3.7 


19.7 


27.1 


6.4** 



* p < .05; ** p < .01 

Note: Differences in group means for institutional type were estimated using one-way ANOVA. Differences in 
group means for institutional control were compared using t test for independent samples. 



special student populations compared to the other institutional types for all four special student 
populations. This finding may reflect either the fact that their students are the most diverse — 
therefore all are special — or their general pattern of devoting less attention to student assessment. 

Public and private institutions differed statistically on three of the four special student 
populations. In all three cases, private institutions gave more attention to these special populations 
than their public counterparts: adult (20.6%), part-time (7.2%), and distance education (27.1%). 

Summary. Overall, institutions seem not to be adopting different assessment methods for 
special smdent populations. While there are few differences among institutional, types, associate of 
arts institutions consistently give the least attention to this area. Despite state pressures for smdent 
assessment, it appears private institutions still give greater attention to using different methods for 
special populations. 



3.6 Smdent Assessment Studies and Reports 
As the practice of smdent assessment has grown, institutions have increasingly looked for 
ways to transform that data into useful information. The extent to which institutions use smdent 
assessment data to conduct studies of their students and to prepare reports for institutional use is an 
important dimension of the institution’s approach to smdent assessment 
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3.6.1 Studies of Student Assessment 



Table 3.12 reports the frequency with which responding institutions reported doing nine 
different types of studies of the relationship between certain student experiences or institutional 
practices and student performance. What stands out are the two highest responses: studies of the 
relationship between admissions standards/policies and student performance (42.1%) or no studies 
(37.5%). The two least reported types of studies were those relating student-faculty interaction 
(14.1%) and classroom, library and/or computing resources (16.6%) to student performance. 
Between 20-30% of the responding institutions reported doing studies on the relationship of the 
following to student performance: course taking patterns (25.6%), different instructional and 
teaching methods (21.4%), extra-curricular activities (23.8%), residence arrangements (21.2%), 
financial aid or employment (29.7%), and academic advising (25.9%). At this time, a significant 
number of institutions either do no studies or only those related to admissions, about one-fifth to 
one-third focus on student academic or other institutional experiences, and very few address the 
key area of student-faculty interaction and the area of educational resources. This latter result is 
surprising given the extensive research on the impact of student-faculty interaction on student 
performance and the growing use of educational technology. 

3.6.2 Studies Bv Institutional Type 

Table 3.12 also portrays the degree to which different types of institutions conducted 
differing kinds of studies. Institutional types differed to a statistically significant degree on how 
they study the relationships between student experiences and performance. Associate of arts 
institutions were most likely to be doing no studies (44.7%) and research universities most likely 
to be doing some (16.0% do no studies). This low level of activity by the associate of arts 
institutions may reflect the relative lack of development of institutional and educational research 
offices, which are usually present in most research universities. There were also two types of 
studies on which there were no differences by institutional type: academic advising and classroom, 
library, and computing resources. 
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Institutions (%) Conducting Studies 



C 

o 

U 



< 

X) 

■o 

u 



c 

o 

U 



co 

c 

O 

E 

V) 



CO 

«s 

fO 

s 

QQ 

H 





4> 


* 










* 


* 






# 














CO 

3 


* 










* 


* 






# 








* 






CN 








Os 


rn 


so 


CN 




00 




tj; 




p 






a* 


00 




m 






00 




O 




in 












o 


CO 

i 














m 


















km 


JC 
































c 

o 


U 
































U O 


































— ^ 

CO rn 

II 


« « 


•— 




vo 




Os 




m 


o 




CN 




°i 




p 




•c ■ 

CL z 


mm 




00 




•n 


o 


Os 


d 




Os 




vd 


r-*‘ 


rn 




2 Z 


CN 










m 


CN 


m 








CN 




m 






































V) 


































C 


u m 


































CN 




mm 






o 




vo 




o 




rn 


m 


°°. 




vr 




X> 00 

3 JI 

Z 


00 




m 




rn 


o 


VO 


Os 




00 




in 


vd 


Os 






CN 




CN 






CN 




CN 




m 




CN 




m 






u 


* 








* 


* 


* 


♦ 




* 








* 






E 


* 




* 




* 


# 


* 


* 




* 








* 






co 


r- 








00 


m 


VO 


«n 








so 










& 


rn 




mm 




CN 


d 


vd 


oo 








CN 


vd 


w 






CO 

1 


mm 








CN 


m 


m 






00 








m 






































J= 


































U 


































•c 


































H in 
«3 


rn 




O 




rn 


© 


rn 


m 




p 






m 


p 






$ JI 










OS 


vd 


rn 


OS 




^r 




vd 


Os 


vd 












CN 




CN 


m 


m 


rr 




VO 




CN 










as 
































a 


^ ^ * ’ 
































£ 


2 

O VO 






© 




m 


in 


vo 


«n 




rn 




p 


SO 






— NT 


o JI 


rr 




m* 




© 




d 






vd 




in 


m 


oo 




CO VO 
C CN 

o 


O 2 

Q 


m 




CN 




CN 


m 




m 




m 




CN 




CN 




•- II 


































3 2 


CO 


































V CN 
































«■* 

CO 


S O 


CN 




Os 




CN 




*— • 


m 




r- 




OO 


CN 


00 




c 


s T 
|z 


tt 




oo 




rn 


d 


CN 


d 








oo 




Tr 






CN 










m 


m 


m 




m 




CN 




m 






. 4> -<r 

*52 


p 










m 


vo 


o 




r^- 




VO 


oo 


CN 






o P m 

« § II 

ffl ^ 


CN 




vo 




vo 


d 


VO 






Os 




vd 


vd 








CN 




m 






m 


CN 


CN 




TT 




CN 




m 






u- 


































O ' OV 














■ - 












? •* 








o 2 - 


© 




© 




VO 


m 


O 


CN 




TT 




Os 


mm 


r- 






o ti «n 

s<4 
< 2 


vd 




•d 




o 


TT 


vd 






r-‘ 




rn 


Os 








CN 




CN 










CN 




CN 




CN 










Os 


































CN 
= m 


vq 




rj; 




«— 


00 


CN 


r- 




-- 




p 


VO 


in 






< T 


in 








Tf 


rn 




Os 




CN 




in 


vd 


r^‘ 






^ II 

2 


CN 




CN 






CN 


CN 


CN 








CN 




m 






c 




























o. 


co 

0 ) 

u 

e 

4> 




8 s 

« 

4> ^ 4> 






00 




"a 


V) 

.2 


S 


ha 

4 

c 




ha 

o 




co 

E 

4) 

ts 


ha 


IE 

co 

C 

o 




mgg 

a. w « 

7 u h 

c 2 & 

o C X 

•2 Ew 


00 

.s 

is 

mm 

1 

4> 

V) 

ha 




s 

<** 

c «c 

4> O 

J- 03 
,4> 4 ) 

isr 

•o o 




Patterns of student-fac 
interaction 


;> 

mm 

o 

cs 

ha 

3 

3 


C 

« 

£ 

4) 

00 

c 

g 


CS 

12 

cs 

is 

*o 

c 


c 

0 

E 

1 
"EL 
E 


0 ) 

"2 

CS 

T5 

C 

CS 

CO 




CQ 

Q. 

00 

.£ 

> 

•o 


>. 4> 

2 3 

•O o 

— CO 
4) 


cs & c 

2 S 1 

Ji > o 

£ et 

>v « St 




cs c 2 oo 

T3 £ 

*£ 5 

« V o 
.Si ^ U. 

•o a 

3 co 


3 

O 

O 

<** 

c 

4) 

•o 

S 

CO 


patterns 


2 "« 
2 J 

5 o 

as 

X o> 

tU .£ 


methods 


O 

fc 

3 

0 

1 

a 

u 


cs 

o 

O 

c 

4) 

!2 

*w 

* 


cs 

c 

c 

c 

o 

•o 

2 

CO 


o 

<h* 

c 

fc 

3 

O 

c 

o 

O 


e 

*2 

*S5 

co 

E 

■u 

< 


policies 


CS 

0 

1 

4) 

T3 

CS 

u 

< 


E oo 

II 

» E 
o 8 


•o 

E 

co 

O 

c 

o 

Q 


4> M 

X c 
^ 0 > 
C T3 

w s 

fl) c 

3 T3 
£ § 
































d 






CO 


— 




<N 




rn 




m 


vo 




r- 




oo 


Os 








70 



82 *P<-05; ** p < .01 



However, on seven of the nine types of studies, there were statistically significant 
differences by institutional type. Given their high rate of not conducting studies of student 
performance, it is not surprising that associate of arts institutions were least likely to be conducting 
five of the seven types of studies on which there were institutional differences. However, they 
were most likely (25.0% - along with doctoral and research universities) to conduct studies of 
exposure to different instructional and teaching methods and student performance. This finding, 
no doubt, reflects their focus on teaching as their primary function. 

Surprisingly, baccalaureate institutions, which were most likely to conduct various types of 
student assessment (section 3.1), were not highest among the five institutional types on any of the 
different types of studies. In fact, they were least likely to undertake two studies on which 
institutions differed: student course-taking patterns (22.7%) and exposure to different instructional 
and teaching methods (16.1%). Given these institutions’ predominant focus on undergraduate 
education, this is surprising, but may reflect the lack of development of an institutional or 
educational research function. 

Master’s institutions were neither highest nor lowest on any of the types of student studies. 
Doctoral institutions on the other hand were most likely (along with research universities and 

C 

associate of arts institutions) to do studies of exposure to different instructional or teaching 
methods (25.0%). 

Given their overall emphasis on student studies, it is not surprising that research 
universities were most likely to conduct five of the seven types of student studies on which 
institutions differed: course-taking patterns (41.3%), student-faculty interaction (29.3%), 
residence arrangements (53.3%), financial aid or employment (49.3%), and admiss ions standards 
and policies (64.0%). Furthermore, they were second most likely on two other types: exposure to 
different instructional or teaching methods (24.0% - similar to associate of arts and doctoral 
institutions) and extra-curricular activities (36.0% - similar to doctoral institutions). These results 
may reflect both the recent emphasis on undergraduate education and the well developed offices of 
institutional or educational research at such institutions. 
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3.6.3 Studies Bv Institutional Control 



In comparing public and private institutions (Table 3. 12), publics were more likely to 
report not doing any studies of the relationship between student experience and performance 
(39.8%) than were their private counterparts. However, publics and privates only differed 
statistically in the degree to which they conducted four of the nine types of studies. Privates were 
more likely to do studies related to admissions standards and policies (45.2%), extracurricular 
activities (30.4%), and residence arrangements (29.5%), while publics were more likely to do 
studies of student course-taking-pattems (28.2%). 

3.6.4 Reports and Profiles 

Respondents were asked about the level at which they aggregated information on student 
assessment data for profiling and reporting these results. Responses are presented in Table 3.13. 

For all respondents, the most (by far) reported preparing either institution-wide reports 
(69.2%) or reports aggregated by academic programs on departments (65.3%). Institutions also 
reported giving considerable attention to studies of special populations or subgroups of students 
(45.7%). They gave lesser attention to preparing reports or profiles by school or college (30.6%) 
or by course or groups of courses (35.9%). Only 10:9% of the responding institutions reported 
providing no reports. 

3.6.5 Reports and Profiles Bv Institutional Type 

Comparing the reporting practices across institutional types (see Table 3.13), we found no 
statistically significant differences oh the most common pattern of institution-wide aggregation of 
reports. This similarity in pattern may reflect either common accreditation requirements or a 
tendency to incorporate assessment results in publicity documents. Institutions differed 
significantly at four of five levels of aggregation: associate of arts institutions were most likely 
among the institutional types to provide reports at the course level (45.6%) and least likely to 
provide school or college reports (13.0%). This finding may reflect their emphasis on teaching 
and/or examining different instructional and teaching methods (Table 3.12). Baccalaureate 
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institutions gave little attention to these reports. They were least likely to do reports on special 
populations or subgroups of students (36.6%) and neither least likely nor most likely among the 
institutional types to aggregate reports at any other level. Master’s institutions were more likely 
than the other types of institutions to provide reports at the academic program or department level 
(76. 1 %) and least likely to provide them at the course level (27.2%). Doctoral institutions were 
neither the most likely nor least likely among the institutional types to provide reports at any level. 
Research universities were most likely than other institutional types to provide reports at the school 
or college level (76.9%) or for special populations and subgroups of students (65.4%). 

3.6.6 Reports and Profiles bv Institutional Control 

Public and private institutions did not differ statistically on their reporting at the school or 
college level or at the academic program or department level (Table 3.13). However, at the three 
levels of aggregating reports on which they did differ, public institutions were higher than the 
privates in all three areas: institution wide (71.6%), special populations or student subgroups 
(49.8%), and at the course level (40.9%). This difference may reflect state level pressures for 
reporting and accountability that often focuses on institutional indicators, special student 
populations, and instructional effort and effectiveness. 

Summary. It is clear that while institutions are giving some attention to studying factors 
that influence student performance, there are still many institutions who do not. To the extent 
studies are done, attention is primarily on enrollment management issues related to the influence of 
admissions and financial aid on student performance. The low level of attention by associate of 
arts institutions to such studies (with the exception of exposure to different instructional and 
teaching methods) is vivid and an area for improvement, while the high level of reported activity in 
research universities, often criticized for their lack of attention to undergraduate education, is 
surprising. At present, institutions seem to be emphasizing reporting and profiling student 
performance —describing it — rather than studying what factors influence it Reports aggregated 
on an institution-wide basis and by academic program or department are extensive (69.2% and 
65.3% of all responding institutions, respectively) with little difference by institutional type or 
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control. There is also considerable attention to reporting on special student populations or 
subgroups. Institutional differences would appear to reflect differences in the academic structure 
of the different institutional types and the greater pressure on public institutions to provide 
institutional reports. 
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4. External Influences on Student Assessment 
This chapter examines the influence of external groups as perceived by institutional 
respondents on institutions’ student assessment efforts, the second research question in this study. 
Five conceptual domains of external influence were identified in the literature review: national 
efforts, state-level initiatives; regional and professional accreditation associations; private sector; 
and professional associations. Of these, state and regional accreditation agencies are discussed as 
being most influential on institutions’ student assessment efforts (Aper et al., 1990; El-Khawas, 
1995, Ewell, 1993). Thus, questions regarding these two domains were emphasized in the 
survey. The role of federal agencies, private sector constituents, and professional associations in 
relation to student assessment was addressed to a lesser extent In the following sections, we 
summarize the pattern of perceived external influences on student assessment for all responding 
institutions (research question two) and then examine variations in this pattern by institutional type 
and control (research question five). 



4.1 Perceived State Role 

Prior research has examined the role of state initiatives on student assessment in general 
terms, asking about the existence and influence of state requirements for student assessment (El- 
Khawas, 1990, Johnson et al., 1991). However, the literature proposes specific dimensions of 
such initiatives that may differentially shape institutional approaches to and support for student 
assessment (Aper & Hinkle, 1991; Ewell, 1987a, 1987b, 1990, 1991, 1994; Ewell et al., 1988). 
This survey examined institutional perceptions of state influences on student assessment in greater 
detail. The survey addressed five dimensions of state requirements for student assessment: 
development of state student assessment plans; influence of state requirements for student 
assessment; state reporting requirements for student assessment; state review of student 
assessment; and criteria used in the state review process for student assessment 
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4.1.1 Development of State Assessment Plans 

Institutions were asked to indicate whether their state’s plan for student assessment was 
primarily developed: (a) by state-level officials or (b) through joint consultation between state 
officials and institutional representatives; or (c) that no state plan for student assessment existed. 
Institutional responses are displayed in Table 4.1. 

A slighdy larger proportion of responding institutions reported a state plan for student 
assessment existed (54.0%) than reported a state plan did not exist (46.0%). Plans were more 
likely to have been developed in consultation between state and institutional officials (38.5%) than 
by state officials alone (15.5%). 

Development of State Assessment Plans bv Institutional Type . Statistically significant 
differences in the development of state plans for student assessment were found across types of 
institutions. Of all institutional types, baccalaureate institutions least often reported the existence of 
a state plan, whether developed by state officials (5.1%) or joindy developed between the state and 
the institution (24.2%), and most often reported that no state assessment plan existed (70.7%). 

This reflects the fact that state plans typically do not apply to baccalaureate institutions, most of 
whom are private. The converse was true for associate of arts colleges which are predominantly 
public institutions. These institutions were the most likely to report that a stale-developed (17.4%) 
or jointly developed (44.8%) plan existed and were correspondingly least likely to report that no 
state plan for student assessment existed (37.7%). Master’s, doctoral and research institutions 
were each about as likely to report a state-developed plan (responses ranged from 1 1.7% to 
16.4%). Compared to master’s and doctoral institutions, research universities more often reported 
a jointly-developed state plan (41.7%) and least often reported that no state plan existed (46.7%). 

Development of State Assessment Plans bv Institutional Control . As would be expected, 
public institutions were significantly more likely than private institutions to report that a state- 
developed (17.2% versus 3.4%) or jointly-developed (43.1% versus 5.9%) state assessment plan 
existed, and were significantly less likely to report that no state plan existed (39.8% versus 90.7%) 
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Summary. Slightly more than half of respondent institutions had some form of state plan 
or requirements for assessment. Observed differences in the existence of state plans among types 
of institutions are largely attributable to differences in institutional control. The majority of 
institutions reporting a state plan for assessment perceived the plan to be the result of joint 
consultation between state and institutions. 

4.1.2 Influence of State Assessment Plans 

Institutions with a state plan for student assessment were subsequently asked about the 
influence these requirements had on their undergraduate student assessment activities. Ins titutions 
could select as many of the following impacts of state requirements as were applicable: (a) 
important reason for the institution to initiate student assessment, (b) increased institution’s 
involvement in student assessment, (c) not a factor, and (d) negative influence on student 
assessment. Institutional responses to this question are displayed in Table 4.1. 

Institutions with a state plan most often reported positive influences of state requirements 
on their student assessment activities. Close to half (45.1%) reported state requirements played an 
important role in initiating student assessment and almost two thirds (62.4%) indicated state 
requirements had increased institutional involvement in assessment In comparison, only 21.6% 
of institutions reported state requirements had not been influential and 4.0% reported a negative 
influence. 

Influence of State Assessment Plans bv InstitutionalTvpe . Institutions differed 
significantly in their perceptions of the influence of state assessment requirements on their own 
student assessment activities. The predominantly public associate of arts (69.4%) and research 
institutions (67.6%) were most likely to report state requirements had increased institutional 
involvement in assessment and least likely to report there had been no influence on assessment 
activities (14.9% and 13.5%, respectively). Baccalaureate institutions, on the other hand, were 
least likely to attribute an increase in institutional assessment involvement to state requirements 
(41.9%) and most likely to report no influence from state requirements (46.5%). Master’s and 
doctoral institutions’ responses fell between these extremes. No significant differences among 



institutional types were found with respect to state requirements leading institutions to initiate or 
having a negative influence on student assessment activities. 

Influence of State Assessment P lans bv Institutional Control Compared to private 
institutions, public institutions were significantly more likely to indicate state requirements had 
stimulated the initiation of assessment activities (47.3% versus 10.8%) or increased institutional 
involvement in assessment (65.0% versus 21.6%), and less likely to say state requirements had 
not influenced their assessment activities (18.4% versus 73.0%). 

Summary . A large proportion of institutions perceived state assessment pl ane and 
requirements as having influenced the student assessment activities undertaken. Among types of 
institutions, baccalaureate institutions appear least affected; they were most likely to report not 
being influenced by these requirements. Conversely, associate of arts and, somewhat 
surprisingly, research institutions, were most likely to report being positively influenced by state 
assessment requirements. The existence and influence of state assessment plans is largely 
restricted to public institutions. This would account for differences in perceived state influence by 
institutional type. 

4.1.3 State Rep orting Requirements 

The nature of institutions’ state-level reporting requirements for student assessment are 
thought to influence the extent of institutional support for and use of student assessment (Ewell 
1987b, 1990). Institutions that reported having state plans for student assessment were asked 
which of the following types of information they were required to report to state officials: (a) 
evidence of a student assessment plan, (b) measurement of state-mandated student performance 
indicators, (c) measurement of institutionally-developed student performance indicators, and (d) 
evidence of having used student assessment information. Table 4.2 presents the percentage of 
institutions reporting each type of reporting requirement 

The most common state reporting requirements were evidence of a student assessment plan 
(67.8%) and measurement of state-mandated student performance indicators (64.2%). 
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XgjLlej^ State Reporting Requirements by Institutional Type and Control 
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Approximately half of respondents were required to report evidence of institutional use of student 
assessment information (51.8%) and use of institutionally-devised indicators (49.1%). 

S ^te Reporting Requirements by Institutional Type . Statistically significant differences in 
three of four state reporting requirements were found across types of institutions. In terms of 
providing evidence of a student assessment plan, baccalaureate institutions were most likely 
(88.2%) and research universities were least likely (57.6%) to have this reporting requirement. 
About two- thirds of associate of arts, master’s and doctoral institutions had this requirement. 
Research institutions were most often required to report use of institutionally-devised indicators 
(75.8%); this was less often a requirement for other types of institutions, particularly doctoral 
institutions (41.7%). Baccalaureate institutions were significantly more likely than doctoral 
institutions to have to provide evidence of institutional use of student assessment information 
(73.5% versus 25.0%). Approximately half of the remaining institutional types reported this 
requirement Institutions did not differ significantly in the likelihood of having to provide 
measurements of state-mandated student performance indicators. Responses here ranged from a 
high of 70.8%^(doctoral institutions) to alow of 51.5% (research institutions). 

State Reporting Requirements bv Instit utional Control . As would be expected, public 
institutions weje significantly more likely than private institutions to be required to report measures 
of state-mandated student performance indicators (65.0% versus 37.5%). No other significant 
differences were observed when comparing reporting requirements by institutional control. This is 
partly a function of the small number of private institutions (N=16) included in this analysis. 

Summary. Institutions are most often required to provide evidence of a student assessment 
plan but less often required to provide evidence of using information collected through assessment 
activities. If measures of student performance indicators are required, these are more likely to be 
state-mandated than institutionally-devised. Taken together, this profile of reporting requirements 
is not expected to contribute a great deal to institutional support for or use of student assessment 
In general, baccalaureate institutions provide more types of student assessment evidence to state 
officials than do other types of institutions. Differences across institutional types are most 
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pronounced with respect to reporting institutional use of assessment information. Although 
differences are evident with respect to other reporting requirements, no clear patterns emerge. 

4.1.4 State Evaluation and Review Criteria for Student Assessment 

States vary in terms of whether and how they evaluate institutions’ student assessment 
plans or information (NCHEMS, 1996). The survey asked institutional respondents to report 
whether state officials had reviewed or evaluated institutions’ student assessment plans or 
processes. Institutions that had undergone state review specified if the evaluation: (a) reviewed 
the institution’s student assessment process; or compared the institution’s student performance 
record (b) to its own past student performance, (c) to that of peer institutions, (d) to other 
institutions in the state, or (e) included other elements. Responses to these questions are presented 
in Table 4.3. 

Slightly more than half (55.9%) of all institutions that reported a state plan for student 
assessment also indicated they had undergone a post hoc state review of their student assessment 
plan or process. This review was most often conducted by state officials (42.1%) and less often 
_ by institutional representatives (24.3%) or external reviewers (16.2%). State reviews most often 
included a review of the institution’s student assessment process (67.2%). Comparisons of 
institutions’ student performance records were less common. If utilized, these most frequently 
involved comparing institutions’ student performance to their own past performance (44.4%), 
followed by comparisons with the student performance records of other institutions in the same 
state (38.2%) or with those of peer institutions (35.8%). 

State Evaluation and Review Criteria bv Institutional Type . Compared to other types of 
institutions, associate of arts colleges were significantly more likely to have had some form of state 
review of their student assessment plan or process (63%). There were no significant differences in 
the occurrence of reviews by state officials or institutional representatives across institutional types. 
Associate of arts and doctoral institutions were almost twice as likely as master’s institutions to 
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have had an evaluation conducted by external reviewers (20.0% versus 1 1.5%). Baccalaureate and 
research institutions reported the least frequent use of external reviewers (7.5% and 8.3% 
respectively). 

Although differences in reports of the elements or evaluative criteria employed in state 
reviews of institutions’ student assessment plans or processes occurred across institutional types, 
none were statistically significant. 

State Evaluation and Review Criteria bv Institutional Control . Differences by institutional 
control suggest that private institutions were far less likely to have undergone a state-level review 
than public institutions (41.6% versus 90.3%). Public and private institutions differed with 
respect to having “other” criteria in the state review of their institution’s student assessment plan. 
Private institutions were much more likely than public to have “other” state evaluation criteria 
(66.7% versus 9.0%). However, statistical comparisons on the basis of institutional control are 
influenced by the small number of private institutions who had undergone a state-level review 
(N=6). 

Summary . Over half of institutions reporting a state initiative for student assessment also 
report that some form of state review of student assessment had been conducted. Because 
institutions reporting such reviews were primarily public (N=574) rather than private (N=31), the 
results reflect public institutions’ experiences. Such reviews have most often been conducted by 
state officials and focused on the institutions’ student assessment process rather than student 
performance reports. Associate of arts institutions are more often subject to state reviews than are 
more prestigious institutions. This approach to evaluation may be expected to produce less 
institutional support for and use of student assessment than one that employs institutional 
representatives or monitors the impact of student assessment on student performance. 

4.2 Regional Accreditation Role 

Past studies have revealed regional accreditation agencies to be an increasingly important 
influence on institutions’ decisions to begin or expand student assessment activities (El-Khawas, 
1990, 1992, 1995; Johnson et al., 1991; Muffo, 1992). Differences among regional accreditors 



on their student assessment-related policies and practices have been reported (Cole et al., 1997) but 
the influence of these differences at the institutional level have not been systematic all y examined. 

In order to extend current knowledge of this external domain, this study asked institutional 
respondents about three dimensions of regional accreditation requirements for student assessment: 

(a) whether the institution had experienced an accreditation review requiring student assessment; 

(b) the influence of regional accreditation requirements for student assessment on the institution; 
and (c) the institutional reporting requirements for student assessment required by the regional 
accreditation body. A fourth dimension, institutions’ use of student assessment services provided 
by regional accreditation associations, will be discussed in section 4.3.2. 

4.2.1 Regional Accreditation Review Experience 

Institutions were asked whether they had undergone a regional accreditation review that 
required undergraduate student assessment. Table 4.4 displays institutional responses. 

The majority of respondent institutions (80.3%) had completed a regional accreditation 
review requiring student assessment. There were no significant differences in the occurrence of a 
regional accreditation review requiring student assessment across institutional types. Nor was 
there a significant difference in the proportion of public and private institutions reporting the 
occurrence of a regional accreditation review requiring student assessment 

4.2.2 Institutional Influence of Regional Accreditati on Requirements 

Next, institutions were asked about the influence regional accreditation requirements had on 
their undergraduate student assessment activities. Institutions could select as man y of the 
following impacts of regional accreditation requirements as were applicable: (a) important reason 
for the institution to initiate student assessment, (b) increased institution’s involvement in student 
assessment, (c) not a factor, and (d) negative influence on student assessment. Institutional 
responses to this question are also displayed in Table 4.4. 
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Institutions generally perceived these requirements as having had a positive influence on 
their student assessment activities. Almost four-fifths (79.2%) reported regional accreditation 
requirements had increased institutional involvement in student assessment and almost two thirds 
(63.6%) reported regional accreditation requirements had been a major reason for initiatin g student 
assessment efforts. Only 12.4% felt these accreditation requirements had not influenced their 
assessment activities and less than 1% reported negative consequences. 

Institutional Influence of Regional Accreditation Requirements bv Institutional Type . There 
were statistically significant differences in the influences of regional accreditation requirements 
reported by various types of institutions. Master’s institutions most often reported that regional 
accreditation requirements had influenced them to initiate student assessment activities (72.1%), 
followed closely by baccalaureate (64.9%), associate of arts and doctoral (both 61.9%) 
institutions. Baccalaureate, master’s and doctoral institutions were most likely to identify regional 
accreditation requirements as having increased institutional involvement in assessment (84.1% to 
85.3%) and least likely to perceive these requirements as having been a source of negative 
influence (8.0% to 11.1%). In comparison, associate of arts colleges were slightly less likely to 
report that institutional involvement in assessment had increased as a result of accreditation 
requirements (75.4%) and slightly more likely to report no influence from these requirements 
(14.6%). Research institutions were most distinctive in their responses. Only 39.7% reported 
regional accreditation requirements as an important reason for initiating student assessment, 70.5% 
felt these requirements had increased institutional involvement in assessment, and almost one- 
quarter (24.4%) viewed them as having had no influence. 

Institutional Influence of Regional Accreditation Requirements bv Institutional Control. 
Private institutions were significantly more likely than public institutions to report that regional 
accreditation requirements had contributed to the initiation of student assessment activities (67.4% 
versus 61.4%) and had increased institutional involvement in assessment (84.1% versus 76.5%) 
and significantly less likely to perceive regional accreditation requirements as having had no 
influence on student assessment (8.9% versus 14.5%). 



Summary. Overall, regional accreditation requirements function as an important source of 
external influence on institutions’ student assessment activities. The influence of this domain 
exceeds that attributed to state requirements. Regional accreditation requirements are more often 
viewed as having been a positive influence and less often reported as having been a negative 
influence than are state requirements (see section 4.1.2). Baccalaureate, master’s and doctoral 
institutions report the most positive consequences while research institutions appear to be 
comparatively less affected by these requirements. The likelihood of experiencing positive effects 
on student assessment efforts from regional accreditation requirements is also greater in private 
than in public institutions. 

4.2.3 Regional Accrediting Reporting Requirements 

- Institutions were asked which of the following types of student assessment information 
they were required to report to their regional accreditation agency: (a) evidence of a student 
assessment plan, (b) intended institutional uses of student assessment information, (c) results of 
student assessment, and (d) evidence of having used student assessment information. Institutions 
could indicate if they were unfamiliar with regional accreditation reporting requirements. Table 4.5 
presents the percentage of institutions reporting each type of reporting requirement 

In terms of reporting requirements, institutions overall most often reported that they were 
required to provide evidence of having a student assessment plan or process in place (90.2%), 
followed by actual (77.4%) and intended (72.7%) institutional use of student assessment 
information, and results of student assessment (66.1%). Very few (4.6%) institutions reported 
being unfamiliar with regional accreditation reporting requirements. 

Regional Accreditation Reporting Requirements bv Institutional Type . Compared to all 
other types of institutions, research institutions were significant less likely to have to provide 
evidence of a student assessment plan (76.3%) or intended institutional uses of student assessment 
information (59.2%) to regional accreditors. Associate of arts, baccalaureate, master’s and 
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Table 4.5 Regional Accreditation Reporting Requirements by Institutional Type and Control 
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doctoral institutions were much more likely to indicate having these reporting requirements (89.2% 
to 94.2% reporting having a plan and 72.2% to 78.5% reporting intended uses). A similar pattern 
of institutional differences existed for the requirements of reporting student assessment results and 
evidence of institutional use of student assessment information although these were not statistically 
significant. Finally, research institutions were more likely (13.2%) than all other institutional types 
(3.1% to 4.4%) to report being unfamiliar with regional accreditation requirements for student 
assessment. 

Regional Accreditation Repotting Requirements bv Institutional Control . There were no 
significant differences in the nature of regional accreditation reporting required of public and 
private institutions. This seems to indicate that regional accrediting agencies apply their 
requirements uniformly across the public and private sectors. 

Summary . Like state reporting requirements, regional accreditation associations most often 
require institutions to provide evidence of having a student assessment plan and least often require 
evidence of student assessment results. However, regional accreditors more often ask for evidence 
of institutions’ intended and actual uses of student assessment information. This pattern of 
reporting requirements may be more conducive to building institutional support for and use of 
student assessment than the more mandatory approaches employed by states. With the exception 
of research institutions, there is little variation in the nature of reporting requirements across 
institutional types. Research institutions appear to have fewer reporting requirements from 
regional accreditors. This may be partly a function of their greater unfamiliarity with accreditors’ 
reporting requirements. Regional accreditation reporting requirements do not differ between public 
and private institutions. 



4.3 External Sources of Support 

In addition to posing requirements for the conduct of student assessment activities, external 
constituents may influence institutions’ engagement in student assessment by providing funding 
(Banta & Associates, 1993) or services (Banta, 1991; Mentkowski, 1991) intended to support 
these efforts. Patterns of institutional use and impacts of various forms of external support have 
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not been systematically examined. To that end, the survey addressed two further dimensions: 
institutional use of external grants and of external services for improving their student assessment 
efforts. 

4.3.1 Receipt of External Grants for Student Assessment 

Institutions were asked if they had received grants to improve their student assessment 
practices from any of the following external sources: FIPSE, other federal agencies, state incentive 
programs, private foundations or corporate sources, or no external grants received. Ins titutional 
responses are presented in Table 4.6. 

Overall, most (79.0%) institutions have not received external grants for improving student 
assessment practices. There was little variation in the receipt of grants from the various external 
sources; the proportion of institutions reporting receipt of grants from any external source listed 
ranged between 5.9% and 7.0%. 

Receipt of External Grants for Student Assessment bv Institutional Type , There were 
statistically significant differences in the receipt of external grants from specific sources by 
institutional type. Associate of arts colleges were the least likely type of institution to have received 
grants for student assessment from FIPSE (2.2%) and from private foundations or corporate 
sources (1.8%) but were the most likely to have received grants from other federal agencies 
(10.4%) (e.g., Carl Perkins or Title HI). Compared to other institutional types, a moderate 
proportion of two-year colleges had received grants from state-level sources (7.9%). 

Baccalaureate institutions were least likely to have received grants from state sources 
(3.4%) and most likely to have received grants from private or corporate sources (1 1.0%). Their 
receipt of FIPSE (7.6%) or other federal source grants (4.8%) was in the moderate range. 

Master’s and research institutions had very similar patterns of external grant receipt For 
both, a comparatively high proportion of institutions had received grants from FIPSE (9.0% and 
9.7%) and from state sources (10.0% and 9.7%), but they were least likely among the ins titutional 
types to have received grants from other federal agencies (4.8% and 2.8%). Research universities 
were most likely to have received grants from foundations or corporate sources (12.5%). 
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Doctoral universities were most likely to have received a FIPSE grant ( 1 0.2%), 
followed by grants from other federal agencies (8.5%) and foundations or corporate 
sources (8.5%). They were least likely to have received grants from state sources (3.4%). 

Recerpt of External Grants for Stude n t Assessment bv Institutional Control . As 
would be expected, public institutions were more likely than private institutions to have 
received a student assessment grant from state sources (10.3% versus 1.1%). They were 
also more likely to have received grants from other federal agencies (8.0% versus 4.1%). 
A larger proportion of private than public institutions reported the receipt of grants from 
foundations or corporate sources (10.3% versus 3.4%). 

Svumnary. Whether due to unavailability, ineligibility, or lack of awareness, very 
few institutions have received external grants for student assessment Institutional types ' 
differ in their receipt of grants from specific external sources. Associate of arts colleges 
make greatest use of non-FIPSE federal funds (such as Carl Perkins or Title m grants) and 
state-provided grants; baccalaureate, doctoral and research institutions make greatest use of 
FIPSE and foundation or corporate grants; and master’s institutions have most often 
received student assessment grants from FIPSE or state sources. As would be expected, 
public institutions are more likely to receive federal or state grants while private institutions 
most often receive student assessment grants from private foundations or corporate 
sources. 

4.3.2 Use of External Resources for Student Assessment 

A variety of postsecondary organizations — professional associations (Banta, 

1991; Mentkowski, 1991), regional accreditation associations (Cole et al., 1997), state- 
level agencies (Boyer, Ewell, Finney, & Mingle, 1987; Ewell, 1987c), and consortia of 
institutions (Astin & Ayala, 1987) — provide a range of services intended to support 
institutions’ student assessment efforts. These services include consultation, assessment 
conferences, training workshops, and publications or research reports on student 
assessment. For each type of postsecondary organization, the survey asked institutions 
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which, if any, of these student assessment services they had used. Institutional responses 
are presented in Table 4.7. 

Institutions reported using some types of student assessment services more often 
than others. Institutions most often reported using assessment conferences. Two-thirds 
(66.5%) of institutions had used an assessment conference provided by one or more of the 
postsecondary institutions considered. Assessment publications or research reports were 
the next most frequently used service, with 59.8% of institutions using this form of service 
from one or more of the provider organizations listed. A somewhat smaller proportion 
(51.8%) had used training workshops from one or more providers while consultation 
services from one or more providers were least likely to have been used (32.9%). 

Patterns of institutional usage emerged by the type of postsecondaiy organization 
providing student assessment services. Professional associations and regional accreditation 
associations were the most frequently reported source of student assessment services. 

Close to two-thirds of institutions reported using one or more types of services from 
professional associations (62.1%) and from regional accrediting associations (60.7%). 

Only a third of institutions reported using assessment services provided by institutional 
consortia (34.1%) or state-level agencies (33.4%). 

Summary . Professional associations and regional accreditation associations play a 
comparatively prominent role in supporting institutions’ student assessment efforts. 
Institutions are most likely to use assessment conferences and publications or research 
reports, less likely to use training workshops, and least likely to use consultation services. 
These patterns of usage may reflect institutional preference or service availability. 
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5. Organizational and Administrative Support for Student Assessment 
The third research question addressed in this study examines the organizational and 
administrative support patterns that institutions have developed to promote student assessment on 
campus. The conceptual framework for the study identified five domains of organizational and 
administrative support for student assessment: the institutional support strategy, le a ders h ip and 
governance patterns, assessment management policies and practices, the institutional culture and 
climate for assessment, and the evaluation patterns for student assessment. Institutional culture 
and climate for student assessment were not addressed in this survey. This domain will be studied 
in the next phase of our research program. Given the emphasis on assessment management 
policies and practices in the literature, we discuss this domain separately in the next chapter of this 
report In the sections that follow, the respondents’ institutional support patterns in the remaining 
three domains of organizational and administrative support — institutional support strategy, 
leadership and governance patterns, and evaluation of the student assessment process — will be 
examined. We will first examine responses for all institutions (research question number three) 
and then examine variations by institutional type and control (research question five). 

5. 1 Institutional Support Strategy for Student Assessment 
The questionnaire addressed three dimensions of institution-wide support patterns for 
student assessment: the mission emphasis on, the intended institutional purposes for, and the 
administrative and governance activities that promote student assessment. 

5.1.1 Mission Emphasis on Student Assessment 

Institutions were asked to identify whether their institution’s mission statement explicitly: 
a) emphasizes excellence in undergraduate education, b) identifies intended student outcomes, c) 
refers to student assessment as an important priority, or d) does not mention any of these. They 
could respond “yes” to more than one. Table 5. 1 displays the institutional responses to this 
question. 
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Table 5.1 Institutional Mission Emphasis by Institutional Type and Control 



Institutional Mission Statement Emphases (% in Institutions) 













Institutional Type 
N=1309 






Institutional Control 
N=1376 
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Institutional Mission 


All 


of Arts 


laureate 


Master’s Doctoral Research 


Chi- 


Public 


Private 


Chi 




Statement Explicitly 


N=1376 


N=539 


N=315 


N=313 


N=64 


N=78 


Square 


N=873 


N=503 


Squz 


1. Emphasizes excellence in 
undergraduate education 


81.9 


75.3 


87.6 


88.8 


87.5 


84.6 


35.2** 


78.9 


87.1 


14.3 


2. 


Identifies intended 
educational outcomes for 
students 


52.0 


49.7 


61.0 


53.4 


42.2 


33.3 


24.8** 


46.6 


61.4 


28.0 


3. 


Refers to student assessment 
as important institutional 
activity 


19.3 


21.3 


15.6 


21.7 


20.3 


9.0 


10.9* 


22.5 


13.7 


15.7 


4. 


Does not explicitly mention 
any of above emphases 


10.8 


14.1 


6.7 


7.0 


9.4 


12.8 


17.1** 


12.9 


7.0 


11.9* 



* p < .05; **p<.01 



Most responding institutions indicated their mission statement emphasizes excellence in 
undergraduate education (8 1 .9%). While over half reported that their mission statement identifies 
intended student outcomes (52.0%), less than 20% reported that their mission statement refers to 
the importance of student assessment (19.3%). Only 10.8% of the institutions did not have one of 
these aspects in their mission statement 

Mission Emphasis By Institutional Type . Table 5.1 confirms that differing types of 
institutions differed statistically in their mission emphases on student assessment While all 
institutional types reported a high level of mission emphasis on excellence in undergraduate 
education, associate of arts institutions rather surprisingly were lower than the four year 
institutions (75.3% compared to 85% and above). Baccalaureate institutions were most likely to 
have mission statements identifying intended student outcomes (61.0%), while research 
institutions were least likely to have such statements (33.3%). The importance of student 
assessment was stressed most in the mission statements of associate of arts, master’s and doctoral 
institutions (20.3% to 21.7%) and least in research universities (9.0%). 

Mission Emphasis bv Institutional Control . Table 5.1 shows that public and private 
institutions differed significantly on the three mission statement components. Private institutions 
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were more likely to emphasize excellence in undergraduate education (87.1% vs. 78.9%) and to 
identify intended outcomes (61.4% vs. 46.6%) while public institutions were more likely to 
identify student assessment as an important activity (22.5% vs. 13.7%). 

Summary. Clearly, most institutions emphasize excellence in undergraduate education in 
their mission statements regardless of type or control. The major difference in mission statement 
emphases both by institutional type and control was the likelihood of institutions identifying 
explicit student outcomes. Baccalaureate and master’s institutions were more likely to be explicit 
about intended outcomes than the other institutional types, while research institutions were least 
likely to do so. Private institutions did so more than public institutions although publics 
acknowledged the importance of student assessment as an activity to a higher degree. 

5.1.2 Purposes of Student Assessment 

Six statements regarding institutional purposes for student assessment were identified in the 
literature. Respondents indicated the importance of each of these six purposes for their institutions 
(1= none, X— minor, 3 = moderate, and 4 = very). Table 5.2 presents the mean scores of the 
respondents on these six purposes. 

Three purposes for student assessment were identified by all institutions as most important- 
preparing an institutional self-study for accreditation (3.86), improving the achievement of 
undergraduate students (3.48), and guiding undergraduate program improvement (3.43). 
Considerably lower, but still of minor to moderate importance, were improving faculty 
instructional performance (3.02), meeting state reporting requirements (2.89), and — lowest — 
guiding internal resource allocation (2.71). 

Purposes of Student Assessment bv Institutional Type . As indicated in Table 5.2, there 
were statistically significant differences among the institutional types on all six purposes for 
student assessment. Despite the more limited student assessment activity noted in previous 
sections, associate of arts institutions held three of the six purposes of higher importance than did 
the other institutional types: improving the achievement of undergraduate students (3.50), meeting 
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Table 5.2 Purpose of Student Assessment by Institutional Type and Control 
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state reporting requirements (3.37), and guiding internal resource allocation (2.83). They were 
also among the highest on preparing for institutional self-study for accreditation (3.61). It would 
appear that associate of arts institutions see student assessment as critically important to all areas of 
management — external reporting, resource allocation, and student and faculty improvement. 

Baccalaureate institutions, like other institutions, saw student assessment as very important 
in preparing an institutional self-study for accreditation (3.63). They also rated the following 
purposes higher than the other institutional types: guiding undergraduate academic program 
improvement (3.51), and improving faculty instructional performance (3.08). Not surprisingly, as 
many of these institutions are private, they saw the purpose of meeting state reporting requirements 
(2.30) as unimportant compared to the other types of institutions. 

Master’s and doctoral institutions had similar views on the purposes of assessment As 
with other institutional types, both saw preparing an institutional self study for accreditation (3.67 
and 3.69), improving the achievement of undergraduate students (3.50 and 3.40), and guiding 
undergraduate academic program improvement (3.46 and 3.28) as very important Both types 
were low — although not lowest— on meeting state performance requirements (2.76 and 2.60) and 
guiding internal resource allocation (2.62 and 2.51). These two institutional types seem to see 
student assessment as serving an internal improvement agenda rather than serving political or 
management purposes. 

Research universities, although they rated preparing an institutional self-study for 
accreditation (3.14), guiding undergraduate academic program improvement (3.29), and improving 
undergraduate student performance (3.17) as important, still ranked five of six purposes for 
student assessment lowest among the institutional types in the importance they ascribed to them. 
Their rating of meeting state reporting requirements (2.41) was almost as low as it was for 
baccalaureate institutions. 

Purposes of Student Assessment bv Control . Public and private institutions differed 
statistically on four of the six purposes of student assessment. They did not differ on preparing an 
institutional self-study for accreditation (both rated it high — above 3.5) and guiding internal 



resource allocation (both rated it low — below 2.75). However, public institutions were much 
more concerned about using student assessment to meet state reporting requirements (3.29). The 
privates were more concerned with purposes directed at improvement of undergraduate academic 
programs (3.51), undergraduate student achievement (3.53), and faculty instruction (3.10). 

Summary . Clearly, preparing for accreditation and improving both undergraduate student 
performance and undergraduate academic programs stand out as important purposes for student 
assessment, while accountability in the form of state reporting requirements and guiding internal 
resource allocation are much lower. Among the institutional types, associate of arts institutions see 
student assessment as important for all purposes. Baccalaureate institutions do also— with the 
exception of meeting state reporting requirements. Public institutions place more emphasis on 
student assessment as an accountability device to meet state reporting requirements while, privates 
emphasize its role in improvement 

5.2 Leadership and Governance for Student Assessment 
The survey instrument addressed a series of issues related to institutional governance and 
leadership patterns supporting student assessment Specifically, it focused on institution-wide 
administrative and governance activities, support for student assessment by various groups, and 
the structure and process of institutional planning and policy setting. These areas and the 
institutional differences are discussed next 

5.2.1 Administrative and Gove rnance Activities Promoting Student Assessment 

Institutions have developed or instituted a variety of administrative and governance 
structures and activities designed to promote student assessment on an institution-wide basis. 

Table 5.3 lists seven such structures or activities and whether respondents have introduced these 
structures or activities in their institutions. 

The two most frequently reported structures/activities were a faculty governance committee 
that addresses student assessment issues (57.8%) and student assessment workshops for 
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Table 5.3 Engagement in Administrative and Governance Activities Promoting Student 
Assessment by Institutional Type and Control 



Institutions (%) Engaging in Activities 













Institutional Type 






Institutional Control 












N= 


1044 








N=1097 


Administrative and Governance 




Assoc 


Bacca- 


















Activities That Promote 


All 


of Arts 


laureate 


Master’s Doctoral Research 


i Chi- 


Public Private 


Chi- 




Student Assessment 


N=1097 


N=428 


N=254 


N=261 


N=50 


N=51 


Square 


N=701 N=396 


Square 


i. 


Annual presidential or 
institution-wide student 
assessment initiatives or 
forums 


41.3 


48.1 


32.7 


41.0 


34.0 


27.5 


21.2** 


44.4 


35.9 


7.6** 


2. 


Rewards or incentives for 
academic and student affairs 
administrators who promote 
unit use of assessment 


6.4 


3.7 


5.9 


7.3 


18.0 


17.6 


27.0** 


5.7 


7.6 


1.5 


3. 


Incentives for academic 
units to use assessment 
information in evaluation 
and improvement efforts 


26.6 


23.6 


25.2 


30.7 


38.0 


29.4 


7.9 


29.0 


22.5 


5.4* 


4. 


Assessment workshops for 
academic and student affairs 
administrators 


56.4 


56.5 


48.8 


61.3 


52.0 


66.7 


11.0* 


57.5 


54.5 


.9 


5. 


Board of trustees committee 
that addresses assessment 


12.8 


10.3 


13.8 


16.9 


10.0 


9.8 


7.3 


12.3 


13.6 


.4 


6. 


Faculty governance 
committee that addresses 


57.8 


49.5 


68.9 


63.6 


54.0 


37.3 


37.4** 


52.1 


67.9 


26.1** 




assessment issues 






















7. 


Student representation on 
assessment committees 


33.4 


28.0 


37.4 


41.0 


36.0 


27.5 


14.9** 


32.5 


34.8 


.6 



* p < .05; ** p < .01 

and student affairs administrators (56.4%). (A similar question on workshops for faculty is 
discussed in section 6.5.). Presidential or institution-wide initiatives, forums, or seminars on 
student assessment were conducted by 41.3% of the responding institutions. Incentives to use 
student assessment information were less frequently mentioned; 26.6% rewarded academic units 
that use assessment information in their improvement efforts and 6.4% provided incentives for 
academic or student affairs administrators to promote assessment. Finally, Board of Trustees’ 
committees on student assessment were not common (12.8%). 
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Administrative and Governance Activities bv Institutional Type . Institutional types differed 
in their use of these institution-wide efforts on five of the seven activities. Associate of arts 
institutions, compared to other institutional types, were most likely to use presidential or 
institution-wide activities (48.1%). Additionally, the use of student assessment workshops for 
academic and student affairs administrators (56.5%) and faculty governance committees (49.5%) 
were mentioned quite often. They were least likely among the institutional types to use incentives 
or rewards for administrators to promote student assessment (3.7%). 

Baccalaureate institutions were most likely to use faculty governance committees (68.9%) 
and least likely to use workshops for academic and student affairs administrators. They were 
neither highest nor lowest on the other activities. Compared to the other institutions, baccalaureate 
institutions apparently focus their efforts on the faculty. 

Master’s institutions were neither highest nor lowest in using any of the activities on which 
institutional types differ statistically. However, it is worth noting that they were most likely to 
have student representation on student assessment committees (41.0%). 

Doctoral institutions relied more than other types of institutions on using rewards and 
incentives for academic and student affairs administrators to promote student assessment (18%). 
Although not at a statistically significant difference, they also reported most frequently the use of 
incentives for academic units (38.0%). This willingness to use incentives seems to be a distinctive 
characteristic of these mostly large institutions. 

Compared to other types of institutions, research universities made the highest use of 
assessment workshops for academic and student affairs’ administrators (66.7%). Like the doctoral 
institutions, they were more likely than other institutions to use incentives for academic and student 
affairs adminis trators (17.6%). They were less likely than the other institutional types to use 
presidential or institution-wide initiatives and events (27.5%) and to have student representatives 
on committees (27.5%). 

Administrative and Governance Activities bv Control . When contrasting public and private 
institutions, there were only statistically significant differences on three of the seven activities. 
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Public institutions reported greater use of presidential or institution-wide initiatives and events 
(44.4%) and incentives for academic units to use student assessment (29.0%). Private institutions 
made greater use of faculty governance committees on student assessment. 

Summary . Overall, institutions engage in the use of faculty governance committees on 
student assessment and workshops on the topic for academic and student affairs adminis trators 
most, and are least likely either to use incentives for administrators or to have board of trustee 
committees focused on student assessment. Institutional comparisons demonstrate that associate of 
arts institutions place a heavier reliance on presidential or institution-wide efforts, baccalaureate 
institutions rely more on faculty governance committees on student assessment, and the larger 
master’s, doctoral and research institutions are more likely to rely on incentives to promote the use 
of student assessment information by academic programs and/or among academic and student 
affairs administrators. 

5.2.2 Constituent Support for Student Assessment 

While all constituents in an institution are potentially affected by student assessment, the 
degree to which institutional efforts are supported is often cited as a critical issue. Respondents 
were asked how supportive six different constituents are of undergraduate student assessment 
activities (Scale: 1 = very unsupportive to 5 = very supportive). The responses are summarized in 
Table 5.4. 

Among all respondents, academic affairs administrators were identified as the most 
supportive of any constituent group (4.64), followed by the chief executive officer (4.41), and 
student affairs administrators (4.33). Boards of trustees (3.84), faculty governance (3.80), and 
students (3.33) were lower but all were perceived as somewhat supportive. 

Constituent Support bv Institutional Type . While there were significant statistical 
differences among the institutional types for four of the six constituencies, it is useful to note that 
the rank ordering of them on their degree of supportiveness was virtually the same for all 
institutional types - from academic affairs administrators at the top, followed by chief executive 
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Type and Control 



Extent to Which Group Supports Student Assessment* 











Institutional Type 
N=1304 






Institutional Control 
N=1T70 






Associate 


Bacca- 
















Internal Constituent 
Group 


All 

N=1370 


of Arts 
N=538 


laureate 

N=313 


Master’s 

N=311 


Doctoral Research 
N=65 N=77 


F 


Public 

N=870 


Private 

N=500 




1. Board of trustees 


3.84 

(.93) 


3.87 

(.95) 


3.79 

(.94) 


3.90 

(.90) 


3.68 

(.91) 


3.67 

(.86) 


1.66 


3.87 

(.93) 


3.81 

(.93) 


1.12 


2. Chief executive 


4.41 


4.47 


4.39 


4.42 


4.30 


4.16 


2.78* 


4.44 


4.38 


1.34 


officer 


(.84) 


(.84) 


(.91) 


(.78) 


(.87) 


(.77) 




(.82) 


(.87) 


3. Academic affairs 


4.64 


4.64 


4.69 


4.68 


4.53 


4.35 


4.41** 


4.62 


4.67 


-1.18 


administrators 


(.69) 


(.71) 


(.71) 


(.60) 


(.76) 


(.66) 


(.68) 


(.70) 


4. Student affairs 


4.33 


4.38 


4.33 


4.27 


4.30 


4.29 


.88 


4.35 


4.33 


.46 


administrators 


(.83) 


(.87) 


(.85) 


(.78) 


(.87) 


(.76) 




(.83) 


(.84) 


5. Faculty governance 


3.80 

(.93) 


3.87 

(.94) 


3.83 

(.90) 


3.77 

(.92) 


3.58 

(.98) 


3.45 

(.85) 


4.49** 


3.77 

(.94) 


3.90 

(.89) 


-2.47* 


6. Students 


3.33 

(.74) 


3.40 

(.76) 


3.33 

(.80) 


3.24 

(.67) 


321 

(-74) 


3.22 

(.63) 


2.92* 


3.35 

(-74) 


3.33 

(-78) 


.47 



* P < Q 5 . mm p < 01 ’ UUWiUWU ’ -^uuicwiidi supportive; 3=very supportive 

Note: Differences across group means for institutional type were estimated using one-way ANOVA, Group means 
for institutional control were compared using t test for independent samples. 



officers, student affairs administrators, the board, the faculty and students. Comparing by. 
institutional type, associate of arts institutions rated the support of their chief executive officers 



(4.48), their faculty governance (3.87), and students (3.40) higher than did other institutional 



types. Baccalaureate institutions saw the academic affairs administrators (4.69) as more supportive 
of student assessment than other institutions did. Master’s institutions, like the b accal au re ate 
institutions, rated academic affairs administrators as more supportive than did other institutional 
types. Doctoral institutions did not see any of the constituents as most or least supportive 
compared to other institutions. Finally, research universities, perhaps reflecting their complexity, 
saw all four of the constituent groups— which differed statistically— as the least supportive 
compared to the other institutional types. All four constituent groups, however, were still rated as 
somewhat supportive: academic affairs administrators (4.35), chief executive officers (4.16), 
faculty governance (3.45), and students (3.22). 
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Constituent Support bv Institutional Control . There were no significant differences 



between public and private institutions in terms of constituent support for student assessment, 
except for a .05 level difference on faculty governance. Private institutions described faculty 
governance (3.90) as more supportive than did public institutions. 

Summary . While all constituent groups are seen as supportive, academic affairs 
administrators are consistently seen as most supportive (perhaps a bias since the survey was mailed 
to them!). More importantly, despite a few statistical differences in constituent supportiveness by 
institutional type and control, the rank ordering of the constituents’ supportiveness remains 
essentially the same for all institutional groupings. Only in research universities are all constituents 
seen as less supportive (but still positive) than in other types of institutions. 

5.2.3 Institutional Plan or Policy for Student Assessment 

A set of questions addressed how institutions provide guidance or formal leadership for 

student assessment efforts. An initial question asked whether institutions had one of seven types 

of institutional plans or overall policies for student assessment Those are: 

Formal Centralization : A formally adopted plan or policy specifying undergraduate student 
assessment activities for all academic programs or units. 

Formal Limited Centralization: A formally adopted plan or policy for undergraduate 
student assessment in some academic programs or units. 

Formal Decentralization: A formally adopted institutional plan or policy requiring all 
academic units or programs to develop their own undergraduate student assessment plan. 

Formal Guidance: A formally adopted institutional plan or policy identifying institution- 
wide activities to be conducted by a central committee or office. 

Informal: No institutional plan or policy but academic units or programs are encouraged to 
develop their own undergraduate student assessment activities. 

Emergent: Currently developing a plan or policy for undergraduate student assessment 

None : Do not have an undergraduate student assessment plan or policy. 

Table 5.5 displays the institutional responses to this question about the status of their 

institutional plan or policy for student assessment 




107 




Table 5.5 Institutions with Plan or Policy for Student Assessment by Institutional Type and Control 
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Institutions could select more than one type of plan or policy 
p <.05; ♦*;><. 01 



Given the complexities of institutional plans, respondents were allowed to check more than 
one type of plan (e.g. an institution could report the usage of both a formal guidance and a limited 
centralization approach). From Table 5.5, it is clear that over 65% of the institutions had some 
type of formal institutional plan or policy for student assessment and 13.0% relied on informal 
planning. Only 4.1% had no plan or policy and 16.6% have an emergent plan. The most common 
approach among all respondents was a formal centralized plan or policy (50.0%) followed by 
institutions that have a formal decentralized (39.2%) or a formal guidance approach (38.2%). 
Fewer institutions reported using a formal limited centralization plan (18.7%). 

Plan or Policy bv Institutional Type . Comparing institutional types, there were statistical 
differences among institutional types on four of the seven types of institutional plans. One notes 
immediately that research universities were by far the most likely to report having no plan (19.0%) 
compared to other types of institutions. 

In areas where institutions differed significantly on the nature of their institutional plan or 
policy, associate of arts institutions were one of the two most likely to have a formal centralized 
plan (53.8%) and least likely to have a formal decentralized one (23.9%). This finding is 
consistent with the more centralized and managerial nature of many community college patterns of 
organization. 

Baccalaureate institutions were the other institutional type most likely to have a formal 
centralized plan (54.8%). They also reported a significant reliance on formal decentralization 
(46.2%) and formal guidance (39.9%) approaches to planning. This finding may reflect two types 
of baccalaureate institutions - those with very centralized plans and those with decentralized plans 
or guidance mechanisms. 

Master’s and doctoral institutions differed from the overall pattern and that of the other 
types of institutions by being most likely to have adopted a formal decentralized plan (58.5% and 
56.9%). Although not statistically significant, they also reported more reliance on limited 
centralization and on a formal guidance approach than did other institutional types. This pattern 
may reflect the need to decentralize efforts in larger more complex institutions. 
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Research universities, not surprisingly, were least likely among the institutional types to 
adopt a formal centralized plan (27.8%). They were also most likely to depend on an info rmal 
approach (25.3%). If one combines emergent plans (12.7%) with none at all (19.0%), over 30% 
of research universities reported having no student assessment plan - almost 50% higher than for 
all other types. 

£ lan_orJ > plicy by Institutional Control . Public and private institutions differed statistically 
on their approach to an institutional plan or policy in only two ways. Private institutions were 
more likely than public institutions to have adopted a formal, decentralized plan (45.5%) or to be in 
an emergent or developmental stage of planning (19.4%). WMe formal centralized plans were 
similar for the two types of institutions, privates appeared to be behind the publics in developing 
institutional plans for students assessment and more likely to develop a decent ralized approach. 

Summary- Institutional plans for student assessment vary extensively and a significant 
number of institutions are still developing them. The institutional type differences are somewhat 
striking with associate of arts and baccalaureate institutions depending most on a formal, 
centralized plan while master’s, doctoral, and to a lesser degree, research universities, are likely to 
emphasize formal decentralized planning. Research universities still are the most decentralized, 
informal or unplanned setting. Privates institutions appear to be slightly behind the public 
institutions in student assessment planning— but catching up. 

5.2.4 Institution-Wide Planning Body or Group 

Implicit in the preceding discussion is a structural question that was addressed explicitly; 
i.e. the proportion of responding institutions that have an institution-wide group, committee, or 
task force responsible for ongoing planning or policy setting for undergraduate student 
assessment. Table 5.6 presents the results of that inquiry. 

Among all respondents, 70.4% of the institutions reported the existence of such a body. 
The differences by institutional type indicate they were more likely to exist in associate of arts, 
baccalaureate, and master’s level institutions (72.7%, 72.5%, and 72.8%, respectively); and less 
likely to be found in doctoral (61.5%) and research universities (44.9%). Public and private 
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institutions did not differ in terms of the existence of an institution-wide group. This pattern seems 
to reflect the lower focus on undergraduate education and/or student assessment in the institutions 
emphasizing doctoral level education and research. 

5.2.5 Membership on Institution-Wide Planning and Policy Setting Bodies 

According to the literature on student assessment, another major issue in planning is 
representation on planning and policy setting bodies and involvement in those processes. 
Institutions with an institution-wide planning or policy group were asked to indicate who served on 
this group. Table 5.6 displays this information for eight different campus positions or functional 
groups. 

The two groups by far most likely to be represented were faculty (90.9%) and an academic 
affairs administrator (85.8%). The other two groups represented by more than 50% of the 
respondents were institutional researchers (60.7%) and student affairs adminis trators (54.3%). 

^Considerably lower were representation figures for academic evaluation administrators (23.5%) 

jand student assessment administrators (32.5%). However, the combination of these two with the 

a* 

'‘institutional researchers may suggest a higher proportion of membership of groups with analytic 

A 

(expertise in student assessment. The lower involvement of student affairs staff (32.3%) and 
students (33.1%) indicates this arena is still the province of academic affairs adminis trators, faculty 
and research/evaluation experts. The lowest level of involvement by the chief executive officer 
(13.0%) is also worthy of note; serving on such groups is not a regular part of their role. 

Membership bv Institutional Type . There were statistically significant differences on six of 
the eight groups represented on these institution-wide committees among the institutional types (see 
Table 5.6). Before commenting on each institutional type, it is important to note the higher level of 
membership by an academic affairs administrator (82.5% to 87.5%) and faculty member (78.1% to 
97.5%) in all types of institutions and the low level of involvement by the chief executive officer 
(2.5% to 16.9%) in all of them. 

Associate of arts institutions differed from the other institutional types in their higher levels 
of involvement by student affairs administrators (66.9%), institutional researchers (67.2%), and 



the chief executive officer (16.9%), and their lower level of student involvement (27.8%). This 
higher level of involvement of chief executive officers, student affairs administrators, and analytic 
expertise may reflect the broad array of purposes associate of arts institutions ascribed to student 
assessment (see section 5.1.2). 

Baccalaureate institutions were notable in having the lowest membership among the 
institutional types of institutional researchers (52.9%), student affairs staff (39.9%), and student 
assessment staff (22.4%). The lower level of student affairs involvement is surprising given that 
these institutions are considered to be student-oriented. The lower level of involvement of research 
and analytic experts may represent the fact that these functions are less well developed in smaller 
institutions. 

Master’s institutions did not differ statistically from other types. However, doctoral 
institutions had the highest proportion of faculty (97.5%) and students (40.0%) compared to other 
institutions and the lowest level of chief executive officer involvement (2.5%). This doctoral 
pattern is in sharp contrast with research universities who, while also having low involvement by 
the chief executive officer (3.1%), had the lowest levels of faculty involvement (78.1%). Research 
universities also had the highest involvement by a student assessment administrator (52.0%) - 
probably reflecting their more extensive and specialized research staff. Institutional researcher 
(62.5%) and academic evaluation staff (34.4%) involvement were also high compared to other 
types of institutions. 

Membership bv Institutional Control . Membership on institution-wide bodies was 
statistically different between pubic and private institutions for only three groups. Public 
institutions were more likely to have student affairs staff (61.4%), institutional research staff 
(66.2%), and student assessment staff (38.3%) involved than are private institutions. This 
difference may reflect both a greater administrative focus of public institutions which, on average, 
are larger than private institutions, and public institutions’ response to state pressures for student 
assessment information. 
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Summary . While faculty and academic administrators are the most represented groups on 
institution-wide student assessment bodies, research and evaluation expertise is also well 
represented. CEOs are not involved directly at this level and student participation is still limited. 
Associate of arts institutions give greater representation to CEOs, student affairs staff and 
institutional researchers. Baccalaureate and master’s institutions do not stand out with a uni que 
pattern. Doctoral institutions stress faculty and student participation most while research 
universities have the lowest faculty involvement (but still high) and emphasize the presence of 
varied types of staff with specific expertise in student assessment. 

5.2.6 Executive Responsibility for Institution-Wide Student Assessment 

Formal leadership for institution-wide planning efforts is often placed in the hands of the 
chair of the institution-wide planning group if one exists. The survey asked who was vested with 
such responsibility at each institution. Table 5.7 summarizes their responses. 

From among the six positions or functions provided as responses, institutions reported that 
an academic affairs administrator most often held this responsibility (55.3%), followed by a faculty 
member (31.1%) or an institutional research officer (17.7%). No other official was named more 
than 10% of the time. 

Executive Responsibility Bv Institutional Type . In all institutional types, the academic 
affairs administrator was most often identified as the person with executive responsibility for 
student assessment planning and policy setting. However, there were significant differences 
among the institutional types in three positions - faculty member, student affairs administrator, and 
academic review or evaluation officer. 

Associate of arts institutions more often placed executive responsibility in the hands of a 
student affairs administrator (12.7%) than did the other institutional types (although they still most 
often mentioned an academic affairs administrator or a faculty member). Baccalaureate institutions 
also used an academic administrator or faculty member most often. However, they were least 
likely among the institutional types to select a student affairs administrator (2.7%). This difference 
probably reflects the academic and faculty focus of these institutions. Master’s and doctoral 
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institutions also followed the pattern of selecting academic administrators most often and a faculty 
member second most often; but compared to the other institutions, they were most likely to select a 
faculty member (38.0% and 42.5%, respectively). Research universities, while relying most on an 
academic administrator — like the other institutional types — varied by using an institutional research 
officer or academic review and evaluation officer second (15.6% of the time for each). They were 
also least likely among the institutional types to use a faculty member in this role (9.4%). This 

finding, again, may reflect the reliance on specialized experts in research and evaluation in these 
institutions. 

Executive Responsibility bv Institu t ional Control . Public and private institutions did not 
differ on the top three positions in whom they place responsibility. In descending order, they 
relied on an academic affairs administrator, a faculty member, and then an institutional research 
officer. However, public and private institutions exhibited a statistically significant difference on 
two positions. Public institutions more often used a student affairs adminis trator (9.6%). Student 
affairs administrators were the fourth most frequently named assessment authority figures in public 
institutions and seventh in private institutions. 

Surpmary . Executive responsibility for planning is most often placed in the hands of an 
academic administrative officer in all types of institutions. Faculty are used second most often in 
all but research universities which rely more heavily on institutional research officers and 
review and evaluation officers than do other types of institutions. Student affairs administrators 
serve this role infrequently but are more likely to be used in associate of arts and/or public 
institutions. 

5.2.7 Approval Authority for Student Assessment Plans and Policies 

Approval of plans and policies for any major institutional decision is often complex in a 
higher educational institution; such appears to be the case for student assessment plans and 
policies. Respondents were asked to identify who, among eleven possible positions or groups, 
had authority to give such approval on their campus. They could identify more than one. Table 
5.8 arrays these responses. 



Institutions (%) Assigning Approval Authority for Student Assessment Plan or Policies 
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On average each respondent checked 2.5 positions or groups in response to this question. 
This response pattern indicates that student assessment plans or policies have to be approved by 
multiple sources. As Table 5.8 indicates, the most frequently mentioned were the chief academic 
officer (75.3%) and the chief executive officer (45.4%). Others in descending order were the 
academic senate or a faculty committee (38.5%), the chief student affairs officer (19.7%), an 
institutional research officer (18.2%), and the board of trustees (17.2%). Other positions or 
groups were mentioned by less than 10% of the respondents. Some may note the irony that 
student government was the least mentioned group (1.2%) to approve student assessment plans 
and policies that directly affect them and presumably are designed to benefit them. 

Approval bv Instit utional Type . There were significant differences among the institutional 
types on nine of the eleven response categories. It should be noted that all five institutional types 
cited the chief academic affairs officer most frequently (71.8% to 88.3% of the time), but 
subsequent patterns differed. 

Associate of arts institutions identified their chief executive officer second most frequently 
(56.5%) and chief student affairs officer third (28.2%) — both were the highest frequencies among 
all institutional types. Associate of arts institutions also named their boards of trustees (24.3%) 
more often than did the other institutional types. This pattern probably reflects the important role 
of student affairs and boards of trustees in these institutions. 

Baccalaureate institutions, after the chief academic affairs officer, mentioned the academ i c 
senate or faculty committee second most frequently (52.2%) and significantly more than the other 
institutional types. These institutions are also least likely to mention a student affairs officer 
(13.0%). The pattern of faculty involvement in these institutions continues. 

Master’s institutions were similar to baccalaureate institutions in citing the chief academic 
affairs officer most often (80.3%) and the academic senate or faculty committee second most often 
(47.7%). However, they were not the highest or lowest among the institutional types in 
mentioning any of the other positions. 

o 
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Doctoral institutions continued the pattern of mentioning the chief academic affairs officer 
most often (76.6%) but dropped drastically in their second most frequently mentioned positions — 
academic senate or faculty committee and chief executive officer (both 29.7%). They mentioned an 
academic review or evaluation officer (20.3%) significantly more often than the other institutional 
types. These institutions, more than the previous ones, seem to place heavier reliance on the chief 
academic affairs officer. 

Research institutions once again stand out. They mentioned the chief academic affairs 
officer first (88.3%) among the positions and also highest by a statistically significant margin over 
the other institutional types. They dropped dramatically in their second most frequently mentioned 
positions: chief executive officer and academic senate or faculty committee (both 21.7%), both of 
which were lowest among all institutional types. They were also least likely by a significant 
margin to identify their board of trustees (6.7%) as a source of approval. Like doctoral 
institutions, they seem to focus a great deal of approval authority in the chief academic affairs 
officer. 

A pproval bv Institutional Control . Public and private institutions differed statistically on 
six of the eleven sources of approval. They did not differ on the most frequently cited source of 
approval — the chief academic affairs officer (74.7% and 76.5%) — however, they differed both 
statistically and in order of sequence on the second and third sources. Public institutions 
mentioned the chief executive officer second (49.5%) and the academic senate or faculty committee 
third (32.8%), while private institutions reversed that order, mentioning faculty bodies second 
(48.1%) and the chief executive officer third (38.5%). In other comparisons, public institutions 
required approval more often from a chief student affairs officer (23.8%), their board of trustees 
(19.3%), and from a student assessment officer (12.1%). 

Summary . Approval of student assessment plans and policies is complex and seems to 
involve multiple layers. However, it tends to be concentrated primarily in chief academic affairs 
officers and secondarily in the chief executive officer or an academic senate or faculty committee. 
The institutional types do not vary on the primary sources of approval but do vary in secondary 
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and lesser sources of approval authority. Chief executive officers and student affairs officers play 
a more central role in associate of arts institutions. Academic senates and faculty committees are 
more often mentioned m baccalaureate and master’s level institutions. Doctoral institutions are 
more likely to use a student assessment officer while research universities concentrate the approval 
authority most in the chief academic affairs officer. The public-private differences reflect the 
tendency of public institutions to place greater reliance for secondary approval on the chief 
executive officer, a chief student affairs officer, their boards of trustees and/or a student 
assessment officer while privates rely more on academic senates and faculty committees as a 
secondary source of approval. 

5 - 2 - 8 Operational Responsibili ty for Student Assessment 

Since an institution-wide student assessment plan or process can involve an extensive 
amount of administrative, logistical, and analytic work, operational responsibility for student 
assessment is often focused in a quite different officer or function than is the executive 
responsibility for, or approval authority over, the institutional plan or policies. The survey asked 
responding institutions to indicate from among six alternatives (eight including “other'’ and “no 

one’’) who in their institutions had day-to-day operating responsibility for student assessment The 
responses are presented in Table 5.9. 

The overall pattern of responses clearly indicates that more than one position or office was 
identified in most institutions. The average number of responses was approximately two per 
institution. The most frequently mentioned positions or offices were an academic affairs 
administrator (45.4%) or an institutional research officer (45.3%) followed by a faculty member 
(32.6%), a student affairs administrator (19.6%), a student assessment officer (15.2%), and an 
academic evaluation or review officer (9.1%). This pattern obviously reflects a diverse array of 
positions or offices overseeing student assessment. 

O perating Responsibility by Institutional Type . All five institutional types mentioned either 
an academic affairs administrator or an institutional research officer most often although their order 
varies by institutional type. There were statistically significant differences on four of the six 



positions or offices identified. Associate of arts institutions relied on an institutional research 
officer most (49.4%) followed by an academic administrator (42.0%). Among the institutional 
types, they were most likely to use a student affairs administrator (24.2%) in this role and least 
likely to use an academic evaluation or review officer (5.9%). 

Baccalaureate institutions relied more than other institutional types on an academic affairs 
administrator for operating responsibility (54.3%). But in the second position, they were most 
likely to depend on an institutional research officer (41.9%), followed by a faculty member 
(39.0%). They were least likely to use a student affairs administrator (13.7%). 

Master’s institutions depended about equally on an institutional research officer (47.3%) or 
an academic affairs administrator (44.4%) but also drew heavily, compared to the other 
institutional types, on a faculty member (38.6%). 

Doctoral institutions primarily used an academic affairs administrator (47.7%), an 
institutional research officer (36.9%) or a faculty member (32.3%). But they differed from the 
other institutional types in using student affairs administrators least (13.8%). 

/. Research universities most often used institutional research officers (48. 1%) or an 
academic affairs administrator. (40.5%), followed by a diverse pattern of other positions. They 
were highest among the institutional types in using a student affairs administrator (22.8%), or 
another officer (19.0%) and lowest in using a faculty member (19.0). 

Operating Responsibility bv Institutional Control . Both public and private institutions 
relied primarily on institutional research officers or academic affairs officers for operating 
responsibility for student assessment However, private institutions statistically placed g reate r 
reliance on an academic affairs administrator (50.9%) or on a faculty member (41.1%) than did 
public institutions. Public institutions meanwhile were statistically more likely to use a student 
affairs administrator (21.2%) or a student assessment officer (18.5%). 

Summary . Operating responsibility for student assessment is typically vested in more than 
one person or position. But primarily, it is placed with an academic administrator or an 
institutional research officer and secondarily with a faculty member. Institutional type differences 



are extensive. While the primary patterns hold, associate of arts institutions use student affairs 
administrators most and academic review and evaluation officers least. Baccalaureate institutions 
use an academic affairs administrator most and a student affairs administrator least. Master’s 
institutions use faculty members to a considerable degree. Doctoral institutions are most likely to 
use an academic review and evaluation officer. Research universities, after the two primary 
offices, are least likely to use an academic affairs officer or a faculty member. Public and private 
differences reflect the public institutions’ larger use of student affairs administrators and student 
assessment officers while private institutions rely more than the publics on academic affairs 
administrators or faculty. 



5.2.9 Reporting_Relatjonship of Individual with Operating Responsibility for Student Assessment 

Institutions responding to the survey were asked to whom the individual with day-to-day 
operating responsibility for student assessment reported. Table 5.10 arrays the responses to this 
question. 

Overall, there were only two administrative offices to which the person responsible for 
day-to-day operations of student assessment typically reported: the chief academic officer (56.3%) 
and the chief executive officer (28.9%). The other three offices were each mentioned less than 8% 
of the time. 

Reporting Relationship bv Institutional Type . Despite the predominance of only two 
offices to whom assessment staff reported, there were significant differences among the different 
institutional types on the offices named. Associate of arts institutions identified the chief academic 
and executive offices as two primary reporting places. However, among the institutional types, 
they were most likely to report to the chief executive officer (37.4%) and least likely to report to the 
chief academic officer (42.6%). They were also most likely to report to a chief student affairs 
officer (12.6%). In baccalaureate and master’s institutions, operating officers reported to the two 
most frequently mentioned offices, but were least likely among the institutional types to report to a 
chief student affairs officer (2.0% and 3.0%). Doctoral institutions were the most likely of any 
institutional type to have the individual with operating responsibility report to the chief academic 
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Table 5.10 Reporting Relationship for O perating Da y-to-day Student Assessment Activities by Institutional Type a nd Control 

I Institutions (%) with Operational Reporting Line for Day-to-day Student Assessment Activities 
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officer (76.6%). They were similar to research institutions in that both institutional types reported 
the lowest levels of reporting to the chief executive officer (9.4% for doctoral and 5.6% for 
research). 

Reporting^Relationship by Institutional Control . Both public and private institutions 
identified assessment staff as reporting most frequendy to the chief academic officer but private 
institutions did so more often (63.7%) with a statistically significant difference. While they both 
mentioned chief executive officers equally in second place, public institutions were more likely to 
mention reporting to a chief student affairs officer (10.1%). 

Summary . Individuals with operating responsibility for student assessment report most 
frequently to the chief academic officer in all types of institutions and in both the public and private 
sector. Both public and private and the various types of institutions, except research universities, 
identify the chief executive officer as the second most frequent reporting recipient. Research 
institutions listed the chief student affairs office second. 



5.2.10 Office Providing Faculty Consultation on Student Assessment 

An administrative feature often mentioned in the literature is an office with expertise in 
student assessment that can serve as a consulting service to faculty, administrators, and/or 
academic units interested in improving their practice and use of student assessment Respondents 
were asked if their institutions had such an office. The questionnaire did not delve into the exact 
functions, activities, or resources of such offices. Responses are shown in Table 5.11. 

As the table indicates, 47.3% of all institutions reported the existence of such an office. 
Furthermore, there was a statistically significant difference among institutional types. Such offices 
were most likely to be found in research universities (63.8%), to a lesser degree in master’s and 
doctoral institutions (53.4% and 50.8%), and least often in associate of arts and baccalaureate 
institutions (46.5% and 40.5%). They were also more likely to be found in public institutions 
(50.7%). These offices seem to be found in the large graduate and research institutions that 
typically have a greater array of academic and administrative support staff. They exist less 
frequendy in those institutions whose primary mission is teaching undergraduates. 
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Table 5.11 Existence of Office Providing Faculty. Consultation 
fo£_Using_StudeiU__Assessnient_by Institutional Type and Control 



Institutions with Office Providing Faculty Consultation for Using 
Student Assessment 





N 


% 


All Institutions (N=1371) 


649 


47.3 


Institutional Type (N=1303) 






Associate of Arts (N=540) 


251 


46.5 


Baccalaureate (N=309) 


125 


40.5 


Master’s (N=309) 


165 


53.4 


Doctoral (N=65) 


33 


50.8 


Research (N=80) 


51 


63.8 


Chi-Square 




19.31** 


Institutional Control (N=1371) 






Public (N=874) 


443 


50.7 


Private (N=497) 


206 


41.4 


Chi-Square 




10.85** 



** p < .01 



5.3 Evaluation of the Student Assessment Process 
Evaluation of any institutional function, process or activity in higher education is always 
encouraged. The literature on student assessment reflects this mantra but provides few examples 
or models of it in practice. This discrepancy is not surprising given the fact that most student 
assessment plans, processes and practices have been introduced within the past decade. The status 
of evaluation of student assessment and its elements were addressed in the questionnaire. 

5.3.1 Status of Evaluation of Student Assessment 

Institutions were asked whether they had conducted a formal evaluation, an informal 
evaluation, were currently developing plans for one or were not doing an evaluation of their 
student assessment process. The results are depicted in Table 5.12. 

The status of institutional evaluation of student assessment was widely varied. Most 
institutions had either conducted an informal evaluation (27.4%) or were planning to do so 
(29.2%). Slightly fewer institutions reported doing a formal evaluation (22.2%) or not doing any 
type of evaluation (21 .2%). 
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S t atus of Evaluation by Institutional Type and rWmi From Table 5.12, it is clear that 
associate of arts institutions were most likely to have conducted formal evaluations (26.0%) and 
less likely to report having done none (20.0%). Baccalaureate and master’s institutions were most 
likely to be developing an evaluation plan (32.6% and 31.4%). Doctoral and research institutions 
were least likely to have done a formal evaluation (15.6% and 17.5%); but doctoral institutions 
were highest in having done informal evaluations (34.4%). Research universities were lowest in 
having done either formal (17.5%) or informal evaluations (18.8%) and most likely not to be 
planning one (38.8%). Public and private institutions did not vary statistically in their evaluation 
of student assessment although private institutions indicated developing plans for an evaluation 
more than publics (33.2%). 

S uMnary . Evaluation of student assessment is not yet well developed. Formal and 
informal evaluations have been done by less than 52% of the institutions regardless of type. 
Research universities stand out as having done the least evaluation to date. 



5 - 3 -2 Elements o f an Evaluation of Student Assessment 

Institutions that reported having conducted formal or informal evaluation were asked to 
indicate which of eight elements of their student assessment plan and process were reviewed (See 
Table 5.12). Five elements were reported as reviewed by more than 60% of all respondents who 
had either formally or informally evaluated their assessment plan or process: plans and policies 
(78.7%), achievement of intended objectives (68.6%), decision use of assessment information 
(65.7%), problems in conducting assessment (67.2%), and structure and responsibility (63.1%). 
Both the reliability and validity of instruments (52.0%) and quality of data analysis (48.9%) were 
also mentioned frequently. Only the cost and benefits of student assessment (21 .8%) were not 
widely addressed. 

Elements of Evaluation bv Ins titutional Type . All institutional types gave the most attention 
to plans and policies for student assessment and the least to cost and benefit analysis. There were 
statistically significant differences among institutional types on only four of the eight elements. 
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Baccalaureate institutions examined all four of these elements more than the other types of 
institutions: plans and policies (97.2%), achievement of intended objectives (73.2%), use of 
information in decision making (72.6%), and cost and benefit analysis (31.1%). Doctoral 
institutions, compared to the other institutional types, gave the least attention to these four 
elements. This pattern suggests that baccalaureate institutions conduct the most thorough 
evaluations of student assessment and doctoral institutions the least thorough. 

Elements of Evaluation b y Institutional Control . There were no statistical differences 
between public and private institutions on any of the elements in the evaluation of student 
assessment. 

Summary. Most elements of a good evaluation are given considerable attention by 
institutions that have done formal or informal evaluations of their student assessment plan and 
process. The actual plans and policies are the most frequently cited elements of a review. Among 
institutional types, baccalaureate institutions seem to be conducting the most comprehensive 
reviews and doctoral institutions the least comprehensive. Public and private institutions give 
similar attention to the eight elements of an evaluation. 
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6. Assessment Management Policies and Practices for Student Assessment 
This chapter examines assessment management policies and practices as a specific domain 
of institutional support for student assessment (research question three). Assessment management 
policies and practices are the mechanisms through which institutions support student assessment 
efforts and increase the likelihood of using the student assessment information collected (Ewell 
1988a, 1997; Sell, 1989b). In the literature, assessment management policies and practices 
designed to support the practice of and use of student assessment were identified in seven 
functional areas or dimensions: resource allocation; student assessment information systems; 
accessibility and distribution of student assessment information; student-related policies; 
professional development; faculty evaluation and rewards; and academic planning and review. 
Findings regarding each of these areas or dimensions are summarized in the sections to follow. 

For each dimension, we present responses from all institutions (research question three) and then 
responses by institutional type and control (research question five). 

6. 1 Resource Allocation for Student Assessment 

Scholars have discussed two broad issues with respect to resource allocation policies and 
practices for student assessment: the explicit commitment of institutional resources for student 
assessment activities (Eisenman, 1991; Thomas, 1991) and the linkage between student 
assessment activities and information to the internal resource allocation process (Ewell, 1987a, 
1987b, 1987c, 1988a). The former issue is expected to affect the capacity of an institution to 
conduct comprehensive student assessment activities, while the latter issue is expected to influence 
internal support, uses and impacts of student assessment Accordingly, the ISS A instrument asked 
respondents to indicate which of the following resource allocation policies or practices existed at 
their institutions: (a) explicit budget allocation for student assessment (b) student performance 
indicators used informally to allocate resources to academic units, (c) student performance 
indicators used to competitively allocate resources among academic units, and (d) student 
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performance indicators used as a basis for rewarding improvement in academic units. Responses 
to these questions are displayed in Table 6.1. 



Almost half (49. 1 %) of all institutions have established an explicit budget allocation to 
support their student assessment activities and close to a quarter (22.9%) used student assessment 
information informally in the budget process. However, very few institutions reported using 
student performance indicators to reward academic units in the budget process (3.3%) or to 
competitively allocate resources among academic units (1.9%). 

6.1.1 Resource Allocation for Student Assessment bv Institutional Type 

Statistically significant differences across types of institutions were found in two resource 
allocation practices. Baccalaureate institutions most often reported having an explicit budget 
allocation for student assessment (56.6%), followed closely by master’s (53.2%), associate of arts 
(47.2%) and doctoral (46.2%) institutions while research universities were much less likely to 
have allocated resources for assessment (33.3%). Baccalaureate institutions were the least likely 
(15.1%) institutional type to report informally using student performance indicators to allocate 
resources among academic units; associate of arts (26.4%) and master’s (25.0%) institutions were 
most likely to do so, followed closely by doctoral (21.5%) and research (20.5) institutions, there 
were no significant differences in the use of student performance indicators to reward or 
competitively allocate resources to academic units; the existence of these resource allocation 
practices was uniformly low across all institutional types. 

6.1.2 Resource Allocation for Student Assessment bv Institutional Control 

Public institutions were statistically more likely than private (26.1% versus 17.4%) to 
informally use student performance indicators to determine resource allocations for academic units . 
No other significant differences emerged with respect to public and private institutions’ resource 
allocation policies and practices for student assessment. 

Summary . While no attempt was made to ascertain the proportion or amount of 
institutional resources being committed, survey findings indicate that close to half of all institutions 
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Institutions (%) with Resource Allocation Policy or Practice 
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have established explicit budget allocations to support their student assessment activities. Research 
universities were an exception to this practice, with only a third having done so. To a lesser 
extent, institutions have informally linked student assessment information to the budget process. 
This practice was less common among baccalaureate institutions compared to other institutional 
types, and consequently, less common among private compared to public institutions. More direct 
linkages between student assessment information and budgetary decisions appear to be a rare 
resource allocation practice. 



6.2 Student Assessment Information Systems 



In practical terms, the availability of a computerized and comprehensive infoimation system 
may affect an institution’s ability to collect and analyze student assessment information (Astin & 
Ayala, 1987; Gill, 1993). In addition, institutions have been encouraged to institutionalize 
opportunities for collecting student assessment data by formally scheduling key assessment 
activities into the academic calendar (Duvall, 1994). Institutional respondents were asked: (a) 
whether they had formally scheduled student assessment activities; and whether they had a student 
information system that (b) was computerized and contained student performance indicators; (c) 
could track students over their enrollment; and (d) was integrated with other institutional databases. 
Responses to these questions are displayed in Table 6.2. 

The majority of institutions (57.3%) had included key student assessment activities in their 
academic calendar. In terms of information system capabilities, about two-fifths (41.9%) of 
institutions could track students and slightly more than a quarter (27.7%) had computer systems 
that included student performance indicators, but very few (9.8%) had integrated st uden t 
assessment data with other institutional databases. 



6.2.1 Student Assessment Information Systems bv Institutional Type 

There were statistically significant differences among institutional types regarding student 
information practices and system capabilities. Baccalaureate institutions were most likely to have 
incorporated assessment activities into the academic calendar (64.1%). Associate of arts (58.4%) 
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Institutions (%) with Student Assessment Information System Policy or Practice 

Institutional Type Institutional Control 
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and master’s (55.5%) institutions reported a moderately high occurrence of this practice, while 
doctoral institutions (47.7%) and research institutions (38.5%) were much less likely to have 
institutionalized student assessment in this manner. On average, associate of arts institutions 
reported the most comprehensive and sophisticated information systems available to support 
student assessment; they were most likely of all institutional types to have a student assessment 
database that was computerized (34.2%) and integrated with databases of other institutional 
information (13.6%). In comparison, baccalaureate, master’s and doctoral institutions were less 
likely to have a computerized student assessment database (occurrence ranged from 19.1% to 
24.4%) and to have the capacity to integrate student assessment data with other institutional 
information (ranged from 4.6% to 7.6%). A moderately high proportion of research uni versities 
reported having a computerized database (30.8%) but a very small percent (6.4%) had integrated 
this database with other institutional databases. 

6.2.2 Student Assessment Information Systems bv Institutional Control 

Private institutions were significantly more likely than public institutions to schedule key 
assessment activities into the academic calendar (62.3% versus 54.4%) but less likely to have a 
computerized information system that includes student performance indicators (203% versus 
31.9%). In both instances, observed differences by institutional control are largely reflective of the 
information system practices of baccalaureate institutions. Public and private institutions did not 
differ statistically in the reported existence of student information .sys tems with tracking and 
integrative capabilities. 

Summary . To a large extent, institutions appear to have institution alize d the collection of 
student assessment information. As may be expected, the likelihood of this form of institutional 
support is greatest among undergraduate institutions but declines progressively as one moves up 
the hierarchy of institutional emphasis on graduate education and research. In terms of information 
systems capabilities, all types of institutions report a comparable degree of ability to track students 
over the duration of their enrollment. Greater variance exists in the likelihood of ins titutions 
having computerized student assessment databases and having the ability to integrate student 
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assessment data with other institutional databases such as faculty, curricular and financial data. It 
seems likely that this may constrain the ability of institutions to systematically analyze student 
assessment data and particularly, to examine how student performance may be affected by 
institutional experiences. 

6.3 Access to and Distribution of Student Assessment Information 

Scholars suggest there is a positive relationship between the extent to which assessment 
information regarding the performance of individual students can be accessed by a variety of 
internal personnel and the institutional use of such information (Krueger & Heisserer, 1987; Sell, 
1989b). Once student assessment data have been collected and analyzed, reports summarizing 
assessment results should be widely and regularly disseminated both within and beyond the 
institution (Banta et al., 1996; CSUUL, 1993; Ewell, 1984, 1988a; Jacobi et aL, 1987). To gauge 
current institutional practices regarding the accessibility and distribution of student ass es s m ent 
information, survey respondents were asked (a) to whom student assessment information on 
individual students was available, and (b) to which internal and external constituencies student 
assessment reports were regularly distributed. Table 6.3 displays responses to both questions. 

Student assessment information on individual students was most likely to be available to 
institutional researchers (76.0%), department chairs or program administrators (73.4%), and 
senior academic administrators (71.9%). Institutions were comparatively less likely to male?- such 
information available to faculty advisors (66.4%) and student affairs professionals (57.9%). 

The majority of institutions regularly distributed student assessment reports to academic 
administrators (85.9%). Faculty were the next most frequently reported recipients of reports 
{ 612 %) followed by student affairs professionals (58.4%). Only one-fifth (19%) of institutions 
distributed assessment reports to students and less than 10% did so to the general public (8.2%) 
and employers (4.6%). 



and master s (55.5%) institutions reported a moderately high occurrence of this practice, while 
doctoral institutions (47.7%) and research institutions (38.5%) were much less likely to have 
institutionalized student assessment in this manner. On average, associate of arts institutions 
reported the most comprehensive and sophisticated information systems available to support 
student assessment, they were most likely of all institutional types to have a student assessment 
database that was computerized (34.2%) and integrated with databases of other institutional 
information (13.6%). In comparison, baccalaureate, master’s and doctoral institutions were less 
likely to have a computerized student assessment database (occurrence ranged from 19.1% to 
24.4%) and to have the capacity to integrate student assessment data with other institutional 
information (ranged from 4.6% to 7.6%). A moderately high proportion of research universities 
reported having a computerized database (30.8%) but a very small percent (6.4%) had integrated 
this database with other institutional database 

6 - 2 - 2 Student Assessment In formation Systems bv Institutional rnntrnl 

Private institutions were significantly more likely than public institutions to schedule key 
assessment activities into the academic calendar (62.3% versus 54.4%) but less likely to have a 
computerized information system that includes student performance indicators (20.3% versus 
31.9%). In both instances, observed differences by institutional control are largely reflective of the 
information system practices of baccalaureate institutions. Public and private institutions did not 
differ statistically in the reported existence of student information systems with tracking and 
integrative capabilities. 

gupimary. To a large extent, institutions appear to have institutionalized the collection of 
student assessment information. As may be expected, the likelihood of this form of institutional 
support is greatest among undergradua t e institutions but declines progressively as one moves up 
the hierarchy of institutional emphasis on graduate education and research. In terms of information 
systems capabilities, all types of institutions report a comparable degree of ability to track students 
over the duration of their enrollment. Greater variance exists in the likelihood of institutions 
having computerized student assessment databases and having the ability to integrate student 



6.3.1 Access to and Distribution of Student Assessment Information bv Institutional Type 

Patterns of assessment information accessibility for specific institutional types mirrored the 
general pattern for all institutions, but statistically significant differences were observed among 
institutional types. 

Associate of arts institutions consistently reported the highest levels of information 
accessibility of all institutional types. This difference was most pronounced with respect to student 
affairs professionals (70.4%), institutional researchers (83.3%), and to a lesser extent, faculty 
advisors (71.4%). 

Baccalaureate and master’s institutions fell consistently in the middle range of reported 
availability of individual student assessment information. They came close to associate of arts 
institutions in providing information access to department chairs (71 .1% and 76.0%, respectively), 
senior academic administrators (75.0% and 68.8%), and faculty advisors (65.5% and 62.7%) but 
were less likely to do so for institutional researchers (72.0% and 773 %) and student affairs 
professionals (51.0% and 49.4%). 

Although research institutions were slightly more likely than doctoral institutions to provide 
access to individual assessment information to each of the internal constituencies listed, together 
these two institutional types uniformly reported the lowest levels of information accessibility. This 
was especially so in the case of having information available to student ‘affairs professionals 
(40.0% to 47.4%) and faculty advisors (55.4% to 56.4%). 

All types of institutions reported similar levels of report distribution to students, academic 
administrators and the general public. Statistically significant differences were found with respect 
to report distribution to other constituencies. 

Of all institutional types, associate of arts institutions were most likely to distribute student 
assessment reports to student affairs professionals (67.1 %) and were the second most likely to 
HiggpTninatp reports to students (18.6%) and employers ( 65 %). Baccalaureate institutions had the 
highest level of report distribution to faculty (7 1 .4%) but were less likely than most other types to 
distribute reports to student affairs professionals (51.0%) and least likely to include employers 




169 



138 



(2.0%) among their audiences for reports. Master’s institutions were neither lowest nor highest in 
their distribution of reports to various constituencies. Doctoral institutions reported the lowest 
occurrence of assessment information distribution to student affairs professionals (49.2%) and fell 
midway among all institutional types in terms of distribution to faculty (60.0%) and employers 
(4.6%). Research institutions were significantly less likely than other institutions to distribute 
assessment reports to faculty (41.0%) but were second highest in their distribution to student 

affairs professionals (62.8%) and somewhat surprisingly, were most likely to send reports to 
employers (7.7%). 

6.3.2 Access to and Distribution of Stud ent Assessment Information hv Institutional Control 

Two statistically significant differences were found in accessibility of assessment 
information between public and private institutions. Public institutions were more likely than 
private ones to provide access to assessment information on individual students to institutional 
researchers (78.8% versus 71.2%) and to student affairs professionals (63.1% versus 48.9%). 

Compared to private institutions, public institutions more often distributed student 
assessment reports to student affairs professionals (62.5% versus 51.1%), employers (6.2% 
versus 1.8%) and the general public (1 1.0% versus 3.2%). 

Summary. On the whole, assessment information on individual students seems to be 
accessible to a relatively large proportion of various internal personnel. However, institutions 
more often provided this access to institutional researchers and academic administrators than to 
personnel with direct student contact Associate of arts institutions reported the greatest 
accessibility to individual assessment information while research and doctoral institutions reported 
the least Public institutions generally provided greater access to individual assessment information 
than private institutions. 

Internal distribution of student assessment reports was also most often di rec ted toward 
academic administrators than to faculty and student affairs professionals, and least often included 
students. Despite some significant differences, no consistent pattern of internal report distribution 
emerged among institutional types. Distribution of reports to external constituencies was 
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comparatively low across all institutional types. As may be expected, public institutions were more 
likely than private to distribute assessment reports to employers or the general public. Together, 
these profiles of access to individual student assessment information and distribution of student 
assessment reports suggest that student assessment information is produced pri mar ily for internal 
consumption, and is used more often to inform administrative or policy-related decisions than 
decisions at the level of direct work with students. 



6.4 Student Policies on Student Assessment 



Collecting useful student assessment information depends in large part upon the 
willingness of students to be involved in assessment activities. Practitioners suggest student 
involvement may be increased if students are either required or provided incentives to participate in 
assessment activities (Duvall, 1994; Van Stewart, 1996) and by providing students both with 
information about the purposes of assessment and with individual feedback on assessment results 
(Dixon, 1994; Duvall, 1994). Respondents indicated the extent to which these student-related 
policies existed at their institutions (l=not done at all; 2=done in a few departments; 3=done in 
some departments; 4=done in many departments; 5=done in most departments). Table 6.4 
presents the mean scores of respondents for each of these policies. 

As shown in Table 6.4, institutions moderately used three of the four student-related 
policies included in the survey. Institutions made greatest use of policies requiring students to 
participate in assessment activities (3.77). This was followed closely by providing students with 
information regarding assessment purposes (3.52), and individual feedback on assessment results 
(3.21). Institutions made least extensive use of incentives to encourage students’ participation in 
assessment activities; this policy existed in none to a few departments within institutions (1.87). 



6.4.1 Student Polic ies on Student Assessment bv Institutional Type 

The extent of use of these student policies differed significantly among institutional types. 
Compared to other types of institutions, associate of arts institutions were most likely to provide 
students with individual feedback (3.38) and least likely to provide incentives for participation 
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Table 6.4 Extent of Student Policies on Student Assessment by Institutional Type and Control 

I Extent Policy or Practice Exists at Institution* 
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(1.72); they fell in the middle range on the remaining two policies. Baccalaureate institutions 
reported the most extensive use of policies requiring student participation (4.02), providing 
incentives (2.06) and informing students about the purposes of assessment (3.72), and were 
second only to associate of arts institutions in providing individual feedback on assessment 
performance (3.25). Master’s and doctoral institutions reported neither the highest nor the lowest 
use of any of these policies. With the exception of providing incentives for participation, research 
institutions made the least extensive use of these policies. 

6.4.2 Student Policies on Student Assessment bv Institutional Control 

Compared to public institutions, private institutions reported more extensive use of 
requirements for student participation in assessment (3.88 versus 3.71) and incentives to 
encourage student participation (2.02 versus 1.78). They did not differ significantly in terms of 
providing information on assessment purposes or individual feedback on assessment performance. 

Summary . It seems that many institutions are institutionalizing student assessment by 
making student participation in assessment activities a requirement The widespread use of this 
policy may contribute to the rather limited use of incentives for student participation. Further, it 
appears that institutions are providing many of their students with information about the purposes 
of assessment, and to a lesser extent, about their individual performance. Overall, baccalaureate 
institutions have made the most extensive use of student policies on assessment This finding is 
consistent with their strong student development orientation. They are followed by associate of 
aits institutions, also an institutional type noted for its strong student orientation. Not surprisingly, 
given the focus of this survey on undergraduate student assessment, research institutions reported 
the least extensive use of. these policies. Private institutions make greater use of student policies on 
student assessment than public institutions. 

6.5 Professional Development Policies on Student Assessment 

The literature has mainly focused on the importance of providing faculty with professional 
development opportunities related to student assessment (Banta et al., 1996; Ewell, 1988b). To a 



lesser extent, institutions have been encouraged to involve student affairs personnel in their 
assessment planning and implementation (Erwin, 1991b; Hanson, 1982). Although professional 
development for administrators is rarely discussed, it seems probable that those in academic 
leadership positions may benefit from access to workshops and seminars regarding student 
assessment. Respondents were asked about the extent to which a variety of professional 
development policies related to student assessment existed at their institutions (l=not done at all ; 
2=done in a few departments; 3=done in some departments; 4=done in many departments; 5=done 
in most departments). Mean scores for each policy are displayed in Table 6.5. ANOVAs were 
used to identify statistically significant differences among institutional types and t tests for 
independent samples were used to test for significant differences by institutional control. 

Examining the mean scores for policy use for all institutions reveals that the most 
commonly used professional development policies were providing funds for faculty to attend 
conferences on student assessment (3.08) and offering faculty workshops or consultation on using 
student assessment (2.90). On average, these policies were used in some departments. 

Institutions were more likely to provide assessment workshops for academic administrators than 
for student affairs administrators (2.55 versus 2.22), and to require faculty rather than student 
affairs staff to receive assessment training (2.47 versus 2.22). Institutions were least likely to 
provide faculty various forms of concrete assistance (such as course load reductions or stipends) to 
encourage their use of student assessment (2.00); this practice was reported in only a few 
departments. 

6-5.1 Professional Devel opment Policies on Student Assessment by Institutional Type 

There were statistically significant differences in the extent to which different institutional 
types used each of these professional development policies. Differences were greatest with respect 
to providing faculty with funds for assessment conference attendance and requiring faculty to 
receive assessment training, and were smallest with respect to providing faculty concrete assistance 
with using student assessment. 
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With only one exception, associate of arts institutions reported the most extensive use of 
professional development policies. These differences were most pronounced in their greater 
tendency to provide faculty with funds for assessment conferences (3.41) and require faculty to 
receive student assessment training (2.76), and to provide assessment training for student affairs 
administrators and staff (2.54 and 2.51). Associate of arts institutions were only slightly less 
likely than doctoral institutions to provide faculty with assistance in using student assessment (2.12 
versus 2.15). These responses are consistent with the tendency of associate of arts institutions to 
make comparatively greater use of professional development activities than other types of 
institutions. 

Baccalaureate institutions were the least likely of all institutions to provide assessment 
workshops for student affairs administrators (1.87) and were second highest among institutions in 
their requirement for faculty to receive assessment training (2.48). Otherwise, their pattern of 
responses was not significantly different from those of master’s institutions, an institutional type 
that consistently fell in the middle range of responses. 

Doctoral institutions presented a more varied profile of professional development policy 
use. Together with associate of arts institutions, they made the greatest use of providing faculty 
assistance in using student assessment (2.15). On all other policies, their responses closely 
resembled those of master’s institutions. 

With the exception of providing assessment workshops to student affairs administrators, 
research institutions made the least extensive use of professional development policies. This 
finding is not unexpected given their focus on graduate rather than undergraduate education, and 
their generally low emphasis on institutionally-provided professional development 

■s ' " 

6.5.2 Professional Development Policies on Student Devel opment bv Institutional Control 

Significant differences in policy use existed between public and private institutions but 
these occurred less often and were of smaller magnitude than those observed among institutional 
types. Compared to private institutions, public institutions were more likely to provide assessment 
workshops and training for student affairs administrators (2.37 versus 1.95) and student affairs 
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staff (2.32 versus 2.04); to provide faculty with assistance for using student assessment (2.10 
versus 1.84) and assessment workshops (3.14 versus 2.98); and to a lesser extent, to have funds 
available for faculty to attend assessment conferences (3.14 versus 2.98). Public and private 
institutions did not differ in the extent to which they required faculty to undergo assessment 
training or in their provision of assessment workshops for academic administrators. 

Summary . While professional development policies are used less frequently than student 
policies on student assessment, most institutions reported using them in a few to some 
departments. Institutions were more likely to direct these policies toward faculty than toward 
administrators or student affairs personnel. These differences apparently reflect the tendency of 
institutions to view student assessment as primarily a responsibility of academic affairs and to view 
faculty as playing a pivotal role in using student assessment Institutional types differed 
significantly in their use of professional development policies. The most extensive use occurred 
within associate of arts institutions and least extensive use was within research institutions. 
Comparisons on the basis of institutional control showed that public institutions used most of these 
policies more extensively than did private institutions. This difference largely reflects the 
differences in policy usage between associate of arts and baccalaureate institutions. 

6.6 Faculty Evaluation and Rewards Policies on Student Assessment 



The use of policies regarding faculty evaluation and rewards to promote faculty 
involvement in student assessment is a contentious issue in the literature. On the one hand, 
scholars warn against linking assessment involvement or results with evaluative criteria and 
consequences (Banta & Associates, 1993; Duvall, 1994). On the other hand, such policies are 
viewed as powerful means of signaling to faculty that student assessment is a valued institutional 
activity (CSU1TL, 1993; Jones & Ewell, 1993). Respondents indicated the extent to which a 
variety of evaluative and reward policies existed at their institutions (l=not done at all; 2=done in a 
few departments; 3=done in some departments; 4=done in many departments; 5=done in most 
departments). Mean scores for each policy are displayed in Table 6.6. ANOVAs were used to 
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identify statistically significant differences among institutional types and t tests for independent 
samples were used to test for significant differences by institutional control. 

Compared to policies concerning students and professional development, institutions 
reported much less extensive use of faculty evaluation and reward policies related to student 
assessment. Mean scores for all institutions showed many of these practices did not exist or were 
done in only a few departments. The significant exception here was the frequency with which 
institutions encouraged their faculty to assess student learning. This informal practice was done in 
many departments (3.99). Much less frequently used policies were: considering faculty 
scholarship on assessment in promotion, tenure or salary reviews (2.01); considering faculty 
participation in assessment activity in promotion, tenure or salary reviews (1.99); and considering 
student performance evidence in faculty promotion evaluations (1.84). Institutions were least 
likely to consider assessment skills when hiring faculty (1.68), to publicly rec ognize faculty for 
effectively using assessment (1.58) or to consider student performance evidence in faculty 
evaluations for salary and merit increases (1.56). 



6.6.1 Faculty Evaluation and Reward Policies bv Institutional Type 

Significant differences by institutional type were found in the use of all but one of these 
policies. Institutions differed in the extent to which they encouraged faculty to assess student 
learning and considered faculty scholarship on assessment in performance reviews. Institutions 
did not differ significantly in their use of public recognition for faculty use of assessment 

Associate of arts institutions reported the most extensive use of three evaluation and reward 
policies: encouraging faculty to assess student learning (4.18); considering assessment skills when 
hiring faculty ( 1 .84); and publicly recognizing faculty for using assessment ( 1 .62). They were 
ranked lowest on two policies: considering faculty scholarship on assessment (1.74) or evi den ce 
of student performance (1.41) in faculty evaluations. These results are understandable in view of 
the lack of emphasis on faculty scholarship in associate of arts institutions and their open 
admissions policies. Further, associate of arts institutions rely more heavily on seniority in making 
retention and promotion decisions than do other institutional types. 
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Table 6.6 Extent of Facult y Evalu ation and Reward Policies on Student Assess ment by Institutional Typ e and Control 

I Extent Policy or Practice Exists at Institution* 
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Baccalaureate institutions used three policies more extensively than other types of 
institutions: considering faculty participation in assessment in promotion, tenure or salary reviews 
(2.35); considering faculty scholarship on assessment in promotion, tenure or salary reviews 
(2.27); and considering student performance evidence in promotion evaluations (2.05). They 
ranked second highest in their use of all other policies. 

Master’s institutions were tied with baccalaureate institutions in their consideration of 
faculty scholarship on assessment in promotion, tenure or salary reviews (2.27) and publicly 
recognizing faculty for effectively using assessment (1.66). Their remaining responses fell 
between those of baccalaureate and doctoral institutions. 

Doctoral institutions reported the least extensive use of student performance evidence in 
faculty evaluations for promotion (1.63). Their remaining responses more closely res emb led those 
of baccalaureate and master’s institutions than associate of arts or research ins titutions. 

Research institutions were most likely to consider student performance evidence in faculty 
evaluations for salary and merit increases (1.78). This surprising finding is tempered by the fact 
that this policy was used in none to a few departments. They ranked in the middle of all 
institutional types on their consideration of assessment scholarship in promotion, tenure or salary 
reviews (1.97) and had the lowest overall use of all other evaluation and reward policies. These 
responses reflect the emphasis of these institutions on research rather than teaching or student 
performance, and also the relative autonomy of faculty within these institutions. 



6.6.2 EacultvJEvaluation and Reward Policies bv Institutional Control 

Statistically significant differences in public and private institutions’ policy use were fewer 
and of smaller magnitude than those observed among types of institutions. Compared to public 
institutions, private institutions reported greater use of the following policies: considering student 
performance evidence in promotion evaluation (2.08 versus 1.70); considering faculty participation 
in assessment in promotion, tenure or salary reviews (2.24 versus 1.85); considering faculty 
scholarship on assessment in promotion, tenure or salary reviews (2.14 versus 1.93); and 
encouraging faculty to assess student learning (4.10 versus 3.93); 
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Summary . Consistent with findings in prior research (Cowart, 1990; Steele & Lutz, 

1995), these results indicate that institutions seldom link faculty evaluation and reward policies to 
student assessment. On the whole, institutions seem more willing to link evaluation decisions to 
faculty participation in assessment, whether in the form of scholarship or involvement in 
assessment activities, than to evidence of student performance. This practice is congruent with 
scholars’ recommendations for encouraging faculty involvement in and use of student assessment 
(CSUTTL, 1993; Ewell, 1984, 1988b). Institutional types differed in their use of various policies. 
Associate of arts institutions used public recognition, hiring criteria and encouragement most 
extensively but made comparatively little use of policies regarding faculty performance evaluation. 
The converse was true of baccalaureate, master’s and doctoral institutions. Research institutions 
reported the lowest level of policy use overall. These findings mirror general differences in how 
various institutional types approach faculty evaluation and rewards. Private institutions reported 
more extensive use of these faculty policies than did public institutions. 

6.7 Academic Planning and Review Policies by Institutional Type and Control 

A final dimension of assessment management policies and practices discussed in the 
literature is that of academic planning and review. In order to encourage the use and impact of 
student assessment activities, scholars have recommended that institutions should link information 
collected through assessment with processes for making academic planning decisions (Barak & 
Sweeney, 1995; Ewell, 1984, 1988a, 1997). Respondents were asked to indicate the extent to 
which student performance data were incorporated in academic planning and review processes at 
the level of academic departments or undergraduate programs; general education or core 
curriculum; $nd courses, and in the review and planning of academic support services (l=not done 
at all; 2=done in a few departments; 3=done in some departments; 4=done in many departments; 
5=done in most departments). Mean responses to these questions are presented in Table 6.7. 
ANOVAs were used to identify statistically significant differences among institutional types and t 
tests for independent samples were used to test for significant differences by institutional control. 
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Institutions have made quite extensive use of student performance data in academic 
planning decisions. Mean scores ranging from 3.09 to 3.67 show these policies existed in some to 
many departments. Institutions reported greatest use of student assessment information in 
academic planning decisions at the levels of departments and programs (3.67), general education or 
core curriculum (3.55), and to a lesser extent, individual courses (3.36). They reported least use 
of this information in planning academic support services (3.09). 

6.7.1 Academic Planning and Review Policies bv Institutional Typ e 

There were differences in the extent to which varying institutional types had incorporated 
assessment information into academic planning processes. All were statistically significant but 
none were of very large magnitude. 

Overall, associate of arts and baccalaureate institutions reported the most extensive use of 
assessment information in academic planning and review decisions. Compared to other types of 
institutions, associate of arts institutions reported the most extensive use of assessment information 
in two types of planning decisions: course-level review and development (3.57) and review and 
planning for academic support services (3.22). They made moderately high use of assessment 
information in planning general education or core curriculum (3.61) and department or program 

V. 

level planning (3.65). 

Baccalaureate institutions reported the most frequent use of assessment information in 
general education or core curriculum review (3.72). They had the second highest use of 
assessment information in the other three planning decisions considered. 

Master’s institutions were the highest user of assessment information in department and 
program planning or review (3.72). They were neither the highest nor the lowest among the 
institutional types in their use of assessment information in other academic planning decisions. 

Doctoral and research institutions were comparatively less likely to incorporate student 
assessment information into academic planning processes. Doctoral institutions made the least use 
of assessmen t information in planning academic support services (2.75) and were the next to 
lowest in incorpo ratin g this information in the other three planning processes. Research 



institutions reported the least use of assessment information in three of the four planning processes: 
department or undergraduate program planning or review (3.29), general education or core 
curriculum review (3.04), and course-level review and development (2.84). They were next to 
lowest in using assessment information for academic support service planning (2.78). 

6.7.2 Academic Planning and Review Policies bv Institutional Control 

Public and private institutions did not differ significantly in their use of student assessment 
information in any of these academic planning processes. 

Summary. Compared to policies concerning students, professional development, and 
especially faculty evaluation and rewards, institutions reported more extensive use of academic 
planning and review policies related to student assessment Assessment information is more often 
incorporated into planning and review processes at the level of program or department; general 
education or core curriculum; and course, and is less often used in planning academic support 
services. There were significant differences among institutional types on all four planning 
processes. Associate of arts and baccalaureate institutions were most likely to link assessment 
information with academic planning and review while doctoral and research institutions were least 
likely to do so. These findings are not surprising in light of differences in these institutional types’ 
emphases on undergraduate education. They do suggest that doctoral and research institutions are 
making less use of student assessment tc improve students’ learning experiences. Public and 
private institutions did not differ in the extent to which they used assessment infor mati on in 
academic planning processes. 
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7. Institutional Uses and Impacts of Student Assessment 
This chapter examines the final domain in our conceptual framework — how institutions 
have used student assessment information and how student assessment has impacted institutions. 
From the literature review, three domains of uses and impacts were identified: the use of student 
assessment information in institutional decision making; internal impacts that have resulted from 
student assessment; and external impacts that have resulted from student assessment These 
domains are considered in the sections that follow. We first e xamin e the pattern of student 
assessment uses and impacts for all responding institutions (research question four), and then 
examine variations in uses and impacts by institutional type and control (research question five). 

7. 1 Influence of Student Assessment Information in Institutional Decisions 
The literature suggests many aspects of institutional decision makin g that can poten tiall y 
utilize student assessment data. These include strategic decisions related to academi c pl annin g, 
acade m ic organization and resource allocation, or more focused decisions regar din g curriculum, 
instructional methods, faculty evaluation and rewards and student support services. Respondents 
were asked to indicate the extent to which student assessment information had influenced twelve 
different institutional decision areas (l=no action or influence unknown; 2=action taken, data not 
influential; 3=action taken, data somewhat influential; 4=action taken, data very influential). Table 
7.1 presents the pattern of responses for each institutional decision for all responding institutions. 

Mean scores provide a broad picture of the extent to which institutions have u tilized 
information available from their undergraduate student assessment processes. Means ran ging from 
139 to 2.61 indicate that assessment information has had little or only limi ted influence on 
institutional decisions. Institutions most often reported that assessment had some degree of 
positive influence with respect to the following actions: modifying student assessment plans or 
processes (2.61); modifying student academic support services (236); designing or reorganizing 
acad e mic programs or majors (2.54); modifying general education curriculum (2.47); and 
modifying teaching methods (2.47). To a lesser extent, institutions reported that assessment 
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Decisions regarding following institutional 
actions 
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Missing 


Mean 


SD 


i. 


Revising undergraduate academic 
mission or goals 


44.0 


12.3 


29.1 


10.1 


4.4 


2.06 


1.09 


2. 


Designing or reorganizing academic 
programs or majors 


23.8 


12.1 


45.2 


15.1 


3.9 


2.54 


1.03 


3. 


Designing or reorganizing student affairs 
units 


49.8 


12.8 


25.1 


7.9 


4.3 


1.91 


1.05 


4. 


Allocating resources to academic units 


49.6 


18.7 


23.4 


3.9 


4.4 


1.81 


.94 


5. 


Modifying student assessment plans, 
policies or processes 


22.7 


12.3 


39.5 


20.6 


5.0 


2.61 


1.07 


6. 


Faculty promotion and tenure 


67.4 


13.4 


12.4 


1.7 


5.1 


1.46 


.78 


7. 


Faculty salary increases or rewards 


70.4 


13.4 


9.8 


1.3 


5.2 


1.39 


.73 


8. 


Modifying general education curriculum 


26.0 


14.5 


39.2 


15.8 


4.5 


2.47 


1.06 


9. 


Modifying student out-of-class learning 
experiences 


36.6 


17.3 


32.9 


8.7 


4.5 


2.14 


1.04 


10. 


Creating or modifying distance learning 
initiatives 


56.1 


13.8 


18.4 


5.5 


6.2 


1.72 


.97 


11. 


Modifying teaching methods 


22.5 


16.5 


45.9 


11.0 


4.1 


2.47 


.97 


12. Modifying student academic support 
services 


22.3 


14.1 


43.9 


16.1 


3.6 


2.56 


1.02 



influential; 4 = action taken, data very influential 



information had influenced modifications to student out-of-class learning experiences (2.14) and 
revisions to undergraduate academic mission or goals (2.06). Institutions were least likely to 
report any influence from assessment information on the following actions: designing or 
reorganizing student affairs units (1.91); allocating resources to academic units (1.81); creating or 
modifying distance learning initiatives (1.72); and particularly, faculty promotion and tenure (1.46) 
and faculty salary increases or rewards (1.39). 

Examining the distribution of responses by response category provides a finer grained 
understanding of institutions’ perceptions of their utilization of assessment information. 

Institutions most often reported five actions as having been influenced by assessment information, 
although a comparison of scores for each decision shows this information was much more likely to 
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be somewhat influential than very influential: modifying teaching methods (45.9% and 1 1.0%); 
designing or reorganizing academic programs or majors (45.2% and 15.1%); modifying student 
academic support services (43.9% and 16.1%); modifying student assessment plans, policies or 
processes (39.5% and 20.6%); and modifying general education curriculum (39.2% and 15.8%). 

Conversely, for the seven remaining actions, respondents most often reported that they had 
not made the decision or did not know to what extent assessment information had influenced the 
action or decision: faculty salary increase or reward decisions (70.4%); faculty promotion and 
tenure decisions (67.4%); creating or modifying distance learning initiatives (56.1%); designing or 
reorganizing student affairs units (49.8%); allocating resources to academic units (49.6%); revising 
undergraduate academic mission or goals (44.0%); and modifying student out-of-class learning 
experiences (36.6%). 

There was little variation in the proportion of institutions reporting that assessment 
information had not influenced specific institutional actions. Hie proportion of institutions 
reporting that assessment data had not been influential ranged from a low of 12.1% for designing 
or reorganizing academic programs or majors to a high of 18.7% for allocating resources to 
academic units. 

Summary . This pattern of responses suggests many respondents were unaware of whether 
assessment had been influential or not in shaping institutional actions. When specific decisions 
had been made and the influence of assessment data was known, respondents were much more 
likely to report that this information had been somewhat influential than not influential or very 
influential. Overall, assessment information was more likely to influence decisions regarding the 
assessment process itself, academic planning and classroom-based instructional practices than 
decisions concerning the budget, out-of-class learning experiences and faculty evaluation and 
rewards. 

7.1.1 Influence of Student Assessment Informa tion in Institutional Decisions bv Institutional Type 

Table 12 presents the mean scores and standard deviations of the influence of assessment 
information on e ach institutional action by institutional type and control. ANOVAs were used to 



identify statistically significant differences among institutional types and t tests for independent 
samples were used to test for significant differences by institutional control. 

There were no statistically significant differences among the five institutional types on the 
assessment influences reported for three institutional actions: designing or reorganizing student 
affairs units, modifying teaching methods; and modifying student academic support services. The 
other nine actions all showed significant differences by institutional type but differences were 
generally not large in magnitude. 

Associate of arts institutions reported the most influence from student assessment 
information on the following actions: modifying student assessment plans or processes (2.70), 
allocating resources to academic units (1.88), and creating or modifying distance learning 
initiatives (1.88). They were least likely among the institutional types to report assessment 
information influences on faculty salary increases or rewards (1.30). Remaining responses fell in 
the middle range among institutional types. 

; Compared to other institutional types, baccalaureate institutions cited the most influence 
from student assessment information. They were highest in reported influence on four institutional 
actions: modifying general education curriculum (2.57), modifying student out-of-class learning 
experiences (2.34), deciding faculty promotion and tenure (1.70) and faculty salary increases or 
rewards (1.49). They were second highest on two additional actions: designing or reorganizing 
academic programs or majors (2.61) and revising undergraduate academic mission or goals (2.09). 

Master’s institutions reported the most assessment influence among institutional types on 
two actions: revising undergraduate academic missions and goals (2.16) and designing or 
reorganizing academic programs or majors (2.67). They reported the second highest influence 
scores for all remaining institutional actions. 3 ? 
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Table 7.2 Influence of Student Assessment Information In Institutional Decisions by Institutional Type and Control 

Extent of Influence of Student Assessment Information * 

Institutional Type Institutional Control 

. N=1281 N=1343 
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CO 

iH 






PS NO OO IO 
©n © m © 

~d™d 



rs r- 
On O 




NO ^ 

o 



no 

rn 




m — 
ps’ ~ 





oo ^ oo m 
m on tt o 
(N w (N r 



no 

O © 

m 

a ii 

sz 



NO On 
— O 

<^d 





O On 
NO O 

rs rl 





TT 

«n o 

<^d 



rs m 
cn o 





NO 

to 

PS 



o 

o 



« NO 

to o 
P m 

3 Jl 
.£ 2 



On 

© 

PS 




S' 



rs 
o Z 



NO On 
© © 
ps 



NO ^ 
^ O 

™d 



OO o 



NO 

Os 



o 

© 
™ d 




O P On NO 
ng tno 



o PS 
© © 
rs 



oo ps 

oo o 



00 

JB 

i s 

^ ° 

if® 

ii 
£1 
J i 



E 

*T3 

s 

CO 

«> 

a 

£ 

00 m 

1 a 

•S O 

*5 « 

5-2 
* E 



c 

v 

*T3 



E * 



C 

-E g 

N E 



00 

c 

N 



V 

GO 

O 



N C N o 

k. ♦* 



§ O 

g*l 

*g> 

o s 

00 ^ 
.5 -H 

5| 

co "O 

01 



§ 

£? 

O 



° 2 
oo^s 
c c 
c » 
00 £ 

2 o 
u c 



C V 

w t- 

E vi e 
8 55 2 

« to _ 
or) o ^ 
“ o C 
O GO 



V fc- 



i 

s 

oo 

£ 

a 

o _ 

o S 

<1 



c & g 

u i_ .£ 

■B ? « 

* E 



flj _ 

6C.£ E 
^ o & 
B- 2? 



1 

e 



e 

o 

•5 

b 

GO 



•a 

o 



£ ^ i 



s-s 



„ u 

CO 00 

E. 



£ 

Urn 



~ PS 







—• ©" NO ~ 

m o «r> © 

r^d^d 



V) 

*8 


0 

1 


w 


■o 

as 


E 


u 

GO 


op 




-5 

*§ 

CO 

2 

oo 


c 

V 

ii 


.S 

jb 


H?* 


•S 

o 

s 


o | 

s §- 
«0 




rs 



c 

CO 

a> 

E 

a. 

o 

o 



•= a 



c 

a> 

o 



c 

o 

GO 

** 

c 

.2 

S 

ii 



< 

> 

O 

2 

< 

CO 

* 

a> 

c 

o 

00 






yj 

•J 

CD 



3 




o 

o 

te 

LU 

£Q 



00 

oo 



f 



158 



Doctoral institutions reported comparatively less influence from student assessment. They 
were least likely to report that student assessment information had influenced decisions regarding 
resource allocations to academic units (1.59). All remaining responses were neither the highest nor 
lowest reported among institutional types. 

Research institutions were least likely of all institutional types to report assessment 
influences. They reported the lowest influence on five institutional actions: designing or 
reorganizing academic programs or majors (2.33); modifying general education curriculum (2.2 6); 
revising undergraduate academic mission or goals (1.51); and deciding faculty promotion and 
tenure (1.32) and faculty salary increases or rewards (1.31). The responses of research 
institutions are not surprising in view of their lesser emphasis on undergraduate education. 

7.1.2 Influence of Student Assessment Information in In stitutional Decisions bv Institutional 
Control 

Public and private institutions differed significantly in reported influence of student 
assessment on four institutional actions. Public institutions reported greater assessment influence 
than private institutions on modifying student assessment plans or processes (2.66 versus 2.52) 
and creating or modifying distance learning initiatives (1.86 versus 1.47). Conversely, private 
institutions reported greater assessment influence than public institutions on modifying student out- 
of-class learning experiences (2.31 versus 2.05) and on faculty promotion and tenure (1.60 versus 
1.37). This latter response mirror the greater tendency of private institutions to have faculty 
evaluation and reward policies related to student assessment 

Summary . The terms “most” and “least” influence must clearly be kept in context in this 
section. Data indicate most institutions have either not used student assessment data to guide 
institutional decisions or were unaware of the influence that student assessment data may have had 
on these actions. Responses in this section reinforce findings in the previous chapter on 
assessment management policies and practices. Overall, institutions have more often used student 
assessment information in the determination of academic planning decisions at the program or 
department, curriculum, and classroom levels, and less often to shape faculty evaluation and 
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reward policies and resource allocation decisions. Baccalaureate institutions, followed by master’s 
institutions, reported the greatest influence from student assessment data on their decisions while 
research institutions reported the least. Differences by institutional control were varied. 

7.2 Internal Impacts of Student Assessment Information 

Although extant research is sparse and conflicting, the literature suggests that information 
collected from student assessment efforts may lead to changes in student performance (CSUITL, 
1993; Johnson et al., 1991) and faculty members’ teaching-related attitudes and behaviors 
(Cowart, 1990; CSUITL, 1993). Respondents were asked whether they monitored the impact of 
student assessment information on a variety of student- and faculty-related performance indicators 
(l=not monitored, do not know; 2=monitored, negative impact; 3=monitored, no known impact; 
4=monitored, positive impact). Table 7.3 presents the pattern of responses for each internal 
performance indicator for all responding institutions. 

Mean scores show that few institutions have monitored the impact of student assessment 
information on these internal institutional indicators. This finding was most pronounced in relation 
to four of the selected indicators: faculty satisfaction (1.69); faculty interest in teaching (1 .88); 
student grade perfor man ce (1.95); and student achievement on external examinations (1.97). 
Institutions were comparatively more likely to have monitored the impact of assessment 
information on teaching methods used (2.45), campus discussions of undergraduate education 
(2.28), student retention or graduation rates (2.20), and student satisfaction (2.03). 

The distribution of responses by response category permits clearer interpretation of the 
meaning of these mean scores. For all the internal impacts listed, the majority of respondents 
reported that they had not monitored the impact of student asse ssme nt information; the percentage 
of institutions selecting this response category ranged from a low Of 44.0% for teaching methods 
used to a high of 64.0% for faculty satisfaction. When institutions had monitored assessment 
information impacts on the indicators, negative impacts were rarely reported. Less than 5% of 
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Table 7.3 Internal Impacts of Student Assessment Information 









%: 


Institutions Reporting Nature of Impact 
of Student Assessment Information* 
N= 1393 






Internal Impacts 


1 


2 


3 


4 


Missing 


Mean 


SD 


1. 


Stimulated campus discussions of 
undergraduate education 


49.7 


1.0 


13.1 


31.6 


4.6 


2.28 


1.38 


2. 


Contributed to faculty satisfaction 


64.0 


4.9 


15.9 


9.6 


5.7 


1.69 


1.08 


3. 


Contributed to faculty interest in 
teaching 


62.0 


1.2 


13.2 


18.7 


5.0 


1.88 


1.25 


4. 


Led to changes in teaching methods used 


44.0 


.2 


15.0 


35.8 


5.0 


2.45 


1.39 


5. 


Contributed to student satisfaction 


54.8 


.9 


20.5 


18.6 


5.2 


2.03 


1.26 


6. 


Affected student retention or graduation 
rates 


47.2 


.8 


27.4 


19.5 


5.2 


2.20 


1.25 


'7. 


Affected student grade performance 


55.6 


.6 


26.6 


12.2 


5.0 


1.95 


1.17 


8. 

ll — 


Affected student achievement on external 
examinations 

1 Ja _ 1 ^ — ^ — 1 _ . 


58.0 


.3 


18.6 


18.2 


5.0 


1.97 


1.25 



*l=not monitored, do not know; 2=monitored,negative impact; 3=monitored,no known impact; 4=monitored, 
positive impact 



respondents had documented a negative impact of assessment infonnation on any of the internal 
indicators considered. On five of the eight indicators, monitoring institutions more often reported 
assessment information had no known impact than a positive impact More than one-quaiter of 
institutions reported monitoring but not observing an impact of assessment information on student 
retention or graduation rates (27.4%) and student grade performance (26.6%), and approximately 
one-fifth had not found assessment-related impacts on student satisfaction (20.5%) and student 
achievement on external examinations (18.6%). 

Approximately one-third of respondents had documented positive impacts of assessment 
infonnation on teaching methods used (35.8%) and campus discussi ons of under g raduate 
education (3 1.6%). Close to one-fifth reported positive impacts on student retention or graduation 
rates (19.5%), faculty interest in teaching (18.7%), student satisfaction (18.6%), and student 
achievement on external examinations (18.2%). Institutions were least likely to have documented 
positive impacts on student grade performance (12.2%) and faculty satisfaction (9.6%). 





161 



Summary . As has been reported elsewhere in the literature, most institutions have not 
monitored the impact of student assessment information on indicators of student performance and 
faculty attitudes and behaviors. When such monitoring has been undertaken, institutions most 
often report assessment information has stimulated discussions of undergraduate education and led 
to changes in teaching methods used. Documentation of positive impacts from assessment on 
direct indicators of students’ academic performance is comparatively less available. 

7.2.1 Internal Impacts of Student Assessment Information bv Institutional Type 

Table 7.4 presents the mean scores and standard deviations of the impact of assessment 
information on each internal indicator by institutional type and control. ANOVAs were used to 
identify statistically significant differences among institutional types and t tests for independent 
samples were used to test for significant differences by institutional control. 

Statistically significant differences among institutional types were found on three internal 
impacts of student assessment information: campus discussions of undergraduate education, 
faculty satisfaction and student grade performance. 

Baccalaureate institutions were most likely to have documented assessment impacts on 
campus discussions of undergraduate education (2.57), followed closely by master’s institutions 
(2.41). Doctoral (2.17), associate of arts (2.12) and research (2.08) institutions were 
comparatively less likely to have monitored and observed this impact 

Baccalaureate institutions were also most likely of the institutional types to have 
documented the impact of assessment information on faculty satisfaction (1.88). Research 
institutions were again least likely to have done so (1.26). Associate of arts (1.71), master’s 
, (1.60) and doctoral (1.56) institutions were in the middle range of responses. 

Somewhat surprisingly, associate of arts institutions were most likely to have documented 
assessment impacts on student grade performance (2.08). Four-year colleges and universities 
were co mp aratively less likely to have documented this internal impact (means ranged from 1.78 to 
1.91). However, faculty satisfaction and student grade performance were the internal impacts of 
assessment least often documented for all types of institutions. 



Nature oflmpact of Student Assessmen t Information* 

Institutional Type Institutional Control 

N=I270 N=1330 
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7.2.2 Internal Impacts of Student Assessment Information bv Institutional Control 



Only two statistically significant differences in documented internal impacts of assessment 
information were found between public and private institutions: campus discussions of 
undergraduate education and faculty satisfaction. 

Compared to public institutions, private institutions had more often documented an impact 
of student assessment information on campus discussions of undergraduate education (2.42 versus 
2.20) and faculty satisfaction (1.78 versus 1.64). The difference in documented internal 
assessment impacts by institutional control was more pronounced for campus discussions than for 
faculty satisfaction. No other statistically significant differences were observed. 

Summary . There are few differences in the tendency of institutional types to have 
monitored and documented internal impacts from student assessment information. Baccalaureate 
and master’s institutions were most likely to attribute an increase in campus discussions of 
undergraduate education to student assessment information. That research and doctoral were 
comparatively less likely to have documented this impact is probably reflective of their lesser 
emphasis on undergraduate education. Given their sole emphasis on undergraduate education,' - 
associate of arts institutions may be less likely to have monitored changes in this institutional 
indicator. Differences in documented impacts on faculty satisfaction and student grade 
performance by institutional type were statistically significant but of little practical significance; 
these indicators were least often monitored by all institutional types. Differences in documented 
internal impacts by institutional control were fewer and smaller in magnitude. For the most part, 
all types of institutions and those under private and public control were more similar than different 
in their tendency to have monitored and documented impacts of assessment information on internal 
indicators. 

7.3 External Impacts of Student Assessment Information 

The litera ture suggests that information collected from student assessment efforts may lead 
to changes in external indicators of institutional performance (Cowart, 1990; El-Khawas, 1995). 



Respondents were asked whether they monitored and documented the impact of student 
assessment information on seven external indicators (l=not monitored, do not know; 
2=monitored, negative impact; 3=monitored, no known impact; 4=monitored, positive impact). 
Table 7.5 presents the pattern of responses on each external performance indicator for all 
responding institutions. 

Table 7.5 External Impacts of Student Assessment Information 





% Institutions Reporting Nature of Impact 
of Student Assessment Information* 

N = 1393 


External Impacts 


i 


2 


3 


4 


Missing 


Mean 


SD 


1 . Affected student application or acceptance 
rates 


74.6 


.7 


12.8 


6.5 


5.4 


1.48 


.97 


2. Affected allocation of state funding 


73.3 


1.0 


10.0 


7.0 


8.7 


1.46 


.96 


3. Affected evaluation from regional 
accreditation agency 


39.3 


2.7 


11.8 


39.6 


6.7 


2.55 


1.39 


4. Affected private fund-raising results 


77.9 


.1 


9.1 


7.0 


5.9 


1.42 


.94 


S. Affected success on grant applications 


69.8 


.2 


10.8 


12.9 


6.2 


1.65 


1.13 


6. Affected communications with external 
constituents 


67.3 


.4 


10.1 


16.7 


5.5 


1.75 


1.21 


7. Affected institutional reputation or image 


60.1 


.6 


13.1 


20.7 


5.5 


1.94 


1.28 



*l=not monitored, do not know; 2=monitored, negative impact; 3=monitored, no known impact; 4=monitored, 
positive impact 



Mean scores show that very few institutions have monitored the impact of student 
assessment information on external institutional indicators. Means for assessment impacts on 
external indicators ranged from 1.42 to 1.94, with only one exception. Institutions were most 
likely to have monitored and documented the impact of assessment information on evaluations 
received from regional accreditation agencies (2.55). In contrast, institutions were much less likely 
to have documented assessment impacts on private fund raising (1.42) and student application or 
acceptance rates (1.48). In addition, comparatively few institutions have documented assessment 
impacts on state funding allocation (1.46). However, this low score reflects the fact that private 
institutions are included in this analysis. 

The distribution of responses by response category allows clearer interpretation of these 
mean scores. Approximately three-quarters of respondents reported they had not monitored the 
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impact of student assessment information on private fund-raising results (77.9%), student 
application or acceptance rates (74.6%), and state funding (73.3%). Three-fifths or more had not 
monitored assessment impacts on grant application success (69.8%), communications with 
external constituents (67.3%), and institutional reputation or image (60.1%). 

When institutions had monitored assessment information impacts on these indicators, 
negative impacts were almost never reported (less than 3.0% for any external indicator 
considered). Institutions were quite consistent in monitoring and yet finding no external impact of 
assessment information; the percentage of institutions who monitored and yet reported no known 
assessment impacts ranged from 9.1% to 13.1% for all eight external indicators. There was 
greater variation in the frequency of institutions reporting positive external impacts of assessment 
Institutions most often reported a documented positive impact of assessment information on 
regional accreditation agency evaluations (39.6%). To a lesser extent institutions reported a 
positive impact on institutional reputation (20.7%). Institutions were least likely to report a 
positive impact from assessment information on student application or acceptance rates (6.5%) or 
funds received from state or private sources (7.0%). 

Summary . Overall, institutions have not monitored the impacts of student assessment 
information on external measures of institutional performance. With one exception, over 60% of 
institutions reported they have not attempted to monitor any these impacts. Only a small proportion 
have documented positive or negative impacts. The striking exception to this general pattern 
concerns regional accreditation impacts. Forty percent of institutions reported that assessment 
information had a positive impact on regional accreditation evaluations. This finding supports the 
important role accorded to regional accreditors as an influence on institutions initiating and 
increasing their assessment efforts. 

7.3.1 External Impacts of Studen t Assessment Information bv Institutional Type 

Table 7.6 presents die mean scores and standard deviations of the impact of assessment 
information on each external indicator by institutional type and control. ANOVAs were used to 
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identify statistically significant differences among institutional types and t tests for independent 
samples were used to test for significant differences by institutional control. 

Statistically significant differences among institutional types were observed on three 
assessment impacts on external indicators of institutional performance: student application or 
acceptance rates, state funding allocations and private fund raising. 

Baccalaureate institutions were the most likely (1.63) and associate of aits institutions the 
least likely (1.40) to have documented impacts of assessment information on student application or 
acceptance rates, with the other institutional types again falling in the middle range of mean scores. 

Research institutions were the most likely (1.66) and baccalaureate institutions the least 
likely (1.24) to have monitored and documented assessment impacts on state funding allocations. 
Associate of aits (1.55), master’s (1.43) and doctoral (1.57) institutions were neither highest nor 
lowest in documenting impacts on this indicator. 

Baccalaureate institutions were most likely to report having monitored and documented 
assessment impacts on private fund-raising results (1.65) and associate of arts institutions were 

least likely to-have done so (1.28). Master’s (1.44), doctoral (1.43) and research (1.41) 

y' 

institutions fell quite squarely in the middle of these response extremes. 

These differences, particularly between baccalaureate and associate of arts institutions, are 
largely attributable to differences in institutional control and admissions practices. However, it 
must be noted that while these differences were statistically significant, all institutional types 
reported low rates of documenting these external impacts. 

7.3.2 External Impacts of Student Assessment Information bv Institutional Control 

Two statistically significant differences by institutional control emerged: allocation of state 
funding and private fund raising. As would be expected, public institutions were more likely than 
private institutions to have documented an impact of assessment information of state funding 
allocations (1.64 versus 1.11). Conversely, private institutions were more likely than public to 
have documented assessment impacts on private fund-raising results (1.57 versus 1.33). Again, 
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while statistically significant* these results reflected little or no attempt to monitor external impacts 
and differed little in practical terms. 

Summary . Institutions have made fewer attempts to monitor and document the impacts of 
assessment information on external indicators of institutional performance than was the case for 
internal indicators. Some expected differences by institutional type and control were found, but 
these were related to external indicators that were seldom monitored by any ins titutions. In 
contrast, institutional types, both public and private, did not differ in their likelihood of 
documenting positive impacts of assessment information on regional accreditation evaluations. 
This external impact of assessment was consistently reported as the strongest 
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8. Data Reduction 



In chapters three through seven, we have presented descriptive profiles of the four domains 
from the conceptual framework: institutional approaches to student assessment, perceived external 
influences on student assessment, organizational and administrative support for student assessment 
(assessment management policies and practices were presented separately as a subsection of this 
domain), and institutional uses and impacts of student assessment These analyses correspond 
with the first five research questions guiding the study. Analyses have focused on item variables 
from the survey inventory organized around these conceptual domains. Results have been reported 
for all respondent institutions, and separately by institutional type and control. 

In the remaining chapters, we examine relationships among variables in the conceptual 
domains of our framework (research questions six through eight). Because of the large number of 
items in the survey inventory, data reduction was used to identify patterns of items in each domain 
and to condense the number of variables used in bivariate and multivariate analyses. The general 
strategy used for data reduction has already been summarized in section 2.5.2. To facilitate the 
reader’s comprehension of the tables and discussion presented in the remaining chapters, this 
chapter describes the data reduction process in greater detail. Two data reduction approaches were 
used: factor analysis and the creation of additive indices in certain sections of the survey inventory. 
The new index variables produced by each data reduction approach are discussed in the next two 
sections. 

8. 1 Index Variables Derived from Factor Analysis 

Factor analyses were used to identify patterns among those survey items constructed on an 
interval scale. Factor analysis was done in two stages. An initial factor analysis of all interval 
variables from across sections of the questionnaire failed to produce clear factor results; drat is, 
items from different sections did not load together on common factors. Separate factor analyses 
were then conducted on items within each section of the inventory. These analysis results revealed 
whether a survey section was comprised of one or more content dimensions. Factors emerged in 



all sections of the inventory. Items were included in a factor if they met three criteria; 1) had the 
highest factor loading on that factor; 2) their factor loading exceeded .40; and 3) the items seemed 
to have content validity. The resulting factors have high Cronbach alpha reliability scores ranging 
from .61 to .84 and individual items have factor loadings ranging from .49 to .90. Indices were 
created for each factor by creating mean scores of the items loading on each factor. All factors 
were scaled in a positive direction. Therefore, the higher the scale value for a factor, the greater the 
extent to which the associated policy or practice existed at an institution. These values were used 
for descriptive analyses. Standardized scores for each factor index were created. A detailed 
description of each factor, the associated survey items, the item factor loading and the Cronbach 
alpha for each factor is displayed in Table 8. 1 . The resultant indices are discussed below by 
survey section. 

8.1.1 Factors Relat ed to Institu tional Approach to Student Assessment 

Three separate factor analyses were conducted on those sections of the inventory related to 
the Institutional Approach to Student Assessment domain. The results follow. 

Extent by Content of Student Assessment Three factors emerged from factor analysis of 
items measuring the type of content and extent of use of student assessment data collection 
practices; cognitive assessment, affective assessment, and postcollege assessment.. These factors 
represent the kinds of student assessment data being collected by institutions and the extensiveness 
(proportion of students on whom information was collected) of these data collection efforts within 
institutions. Cognitive assessment measures the extent to which institutions collect data on aspects 
of students' cognitive or academic performance such as general education competencies, higher- 
order cognitive skills and vocational or professional skills. Affective assessment measures the 
extent to which institutions collect information regarding students’ affective development and their 
satisfaction and experiences with the institution. Postcollege assessment measures the extent to 
which institutions collect information regarding their former students' employment and educational 
experiences and post-enrollment relationship with the institution. Four item variables did not load 



Table 8.1 Factor Analysis Results by Section of Questionnaire 

Section of Survey Factors - Variable Name* Factor Alphft 

Loading Reliability 

I. Institutional Approach 
to Student Assessment 



A. Extent by Content Factor 1 - Cognitive Assessment .71 

IA5 competence in major field .77 

IA4 general education competencies .72 

IA3 higher-order skills .69 

IA6 vocational or professional skills .69 

Factor 2 - Affective Assessment .68 

IA8 student experiences and involvement with institution .81 
IA9 student satisfaction with institution .70 

IA7 personal growth affective development .68 

Factor 3 - Postcollege Assessment .83 

IA1 1 vocational or professional outcomes .89 

IA12 further education .87 

IA 1 4 satisfaction/experiences with institution after . 80 

leaving 



IA1 academic intentions 

IA2 basic college-readiness skills 

IA10 academic progress 

1A13 civic/social roles of former students 



C. Other Student Factor 4 - Student-Centered Methods .61 

Assessment Methods 

IC3 student performance in capstone courses .79 

IC2 student portfolios or comprehensive projects .77 

IC1 observations of student performance .56 

IC4 student interviews or focus groups .5 1 

Factor 5 - External Methods .63 

IC9 employer interviews or focus groups .77 

IC8 alumni interviews or focus groups .74 



ICS transcript analysis 

IC6 external examinations 

IC7 surveysAnterviews with withdrawing students 

♦italicized questionnaire items did not load on factors 
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E. Student Assessment Factor 6 - Curricular Experience Studies 
Studies 

IE2 exposure to different teaching methods 
IE3 patterns of student-faculty interaction 
IE9 classroom, library and/or computing resources 
IE8 academic advising patterns 
IE1 course-taking patterns 

Factor 7 - Co-curricular Experience Studies 

IE5 residence arrangements 
TEA extra-curricular activities 
IE6 financial aid and/or employment 
IE7 admission standards or policies 

II. Organizational and 
Administrative Support 
for Student Assessment 

B. Purpose of Student Factor 8 - Internal Purposes 
Assessment 

DB4 guiding undergraduate academic p ro gram 
improvement 

DB5 improving achievement of undergraduate students 
UB6 improving faculty instructional performance 
KB3 guiding resource allocation decisions 

IIB1 conduct for accreditation 
IIB2 conduct for state 

TV Assessment 

Management Policies 
and Practices 



.69 

.69 

.68 

.65 

.60 



.80 



.73 

.70 

.63 



.85 

.84 

.75 

.71 



E. Student Policies on Factor 9 - Student Involvement 
Student Assessment 

IVE3 students informed about student assessment .79 

purpose and uses 

TVE1 students required to participate in assessment .75 

activities 

IVE4 students provided individual feedback on assessment .75 
results 



TVE2 student incentives 



.69 



.70 



.79 



.69 
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F. Professional Factor 10 - Professional Development .77 

Development 

IVF2 funds for faculty to attend assessment 
conferences 

IVF3 student assessment workshops for faculty 
IVF4 faculty assistance for using assessment 
IVF5 student assessment workshops for academic 
administrators 

Factor 11 - Student Affairs .84 

IVF6 assessment training required for student affairs staff .88 

IVF7 student assessment workshops for student affairs 
administrators .87 

IV 1 faculty training required 

G. Faculty Evaluation Factor 12 - Faculty Evaluation 
and Rewards 

IVG1 promotion evaluation includes student performance 
IVG2 salary evaluation includes student performance 
IVG4 evaluation considers faculty participation in student 
assessment 

IVG3 evaluation considers scholarship on student 
assessment 

IVG5 public recognition for faculty use of assessment 

IVG6 hiring process 
IVG7 encourage faculty to assess 



H. Academic Planning Factor 13 - Academic Planning and Review .84 

and Review 

IVH3 course review uses assessment data .84 

IVH1 department or program planning uses assessment 
data * *. v84 

IVH2 curriculum review uses assessment data .83 

IVH4 academic support service planning uses assessment .76 
ddtd 



.77 

.76 

.73 

.71 

.50 



.76 

.76 

.67 

.66 



ERjt 



212 



174 



V. Uses and Impacts of 
Student Assessment 



A. Decision Making Factor 14 - Academic Decisions .83 

VA1 1 modify instructional or teaching methods .7 1 

VA2 design academic programs or majors .69 

VA8 revise general education curriculum .66 

VA9 create out-of-class learning experiences .66 

VA1 revise undergraduate academic mission .64 

VA12 modify student academic support services .64 

VA5 modify assessment plans or processes .60 

VA3 design student affairs units .58 

VA4 allocate resources to academic units .57 

VA10 create distance learning initiatives .54 

Factor 15 - Faculty Decisions .79 

VA7 decide faculty salary increases .90 

VA6 decide faculty promotion and tenure .90 

B. Institutional Impacts Factor 16 - Faculty Impacts .79 

VB3 faculty interest in teaching .8 1 

VB 1 campus discussions of undergraduate education .75 

VB2 faculty satisfaction .70 

VB4 changes in teaching methods used .63 

Factor 17 • Student Impacts .82 

VB7 student grade performance .89 

VB6 student retention or graduation .83 

VB8 student achievement on external examinations .67 

VB5 student satisfaction .65 

Factor 18 - External Impacts .82 

VB 1 3 success on grant applications .77 

VB 14 communication with external constituents .76 

VB 12 private fund-raising results .75 

VB 15 institutional reputation or image .66 

VB10 allocation of state funding .61 

VB9 student applications or acceptance rates .55 

VB 1 1 regional accreditation evaluations .49 



on any of these factors: students’ academic intentions, basic college-readiness skills, academic 
progress, and civic or social roles of former students. These items were retained as variables in 
subsequent data analyses. 
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Other Student Assessment Methods . Two indices resulted from factor analyses of the 
section concerning institutional use of student assessment methods other than traditional tests or 
examinations: student-centered methods and external methods. Student-centered methods reflects 
the degree to which institutions used assessment methods that require students to demonstrate 
competencies or apply knowledge and skills, such as portfolios of student work, performance in 
capstone courses and observations of student performance. External methods was a two-item 
factor measuring the extent to which units within institutions conducted individual or group 
interviews with alumni or employers of students. 

Student Assessment Studies . Factor analysis of the studies institutions conducted of the 
relationship between students’ institutional experiences and students’ performance produced two 
indices: curricular experience studies and co-curricular experience studies. Curricular experience 
studies measures the extent to which institutions study how various aspects of students’ academic 
experiences such as exposure to different teaching methods, interaction with faculty, learning 
resources, academic advising, and course-taking patterns are related to students’ performance. Co- 
curricular experience studiesmeasuizsihe extent to which institutions conduct studies of the 
relationshipbetween students 7 performance and their non-academic experiences including 
residence arrangements, extra-curricular activities, admission policies, and financial aid or 
employment status. 

’ ' * ’ * - 

8.1.2 Factors Related to Organizational and Administrative Support for Student Assessment 

Factor analysis was conducted on the section of die inventory related to the Organizational 
and Administrative Support for Student Assessment domain. 

Purpose of Student Assessment . Two factors emerged from the analysis of the intended 
purposes of institutions’ undergraduate student assessment activities but only the first of these 
indices was retained. The index internal purposes reflects the importance of four internal 
ins titutional purposes for undertaking student assessment: guiding academic program 
improvement, improving student achievement, improving faculty instructional performance and 
guiding resource allocation decisions. Two external purposes of student assessment loaded on a 
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separate factor: preparing an institutional self-study for accreditation and meeting state reporting 
requirements. However, given the conceptual distinctiveness of these items, a decision was made 
to keep these as separate variables in subsequent data analyses. 

8.1.3 Factors Related to Assessment Management Policies and Practices 

Two factor analysis approaches were used for items in the five subsections of the 
Assessment Management Practices section of the survey instrument. One factor analysis was done 
for all items in this domain and separate factor analyses were conducted for each subsection. 

These two approaches produced identical results. Five factors emerged: they are discussed by 
each subsection of assessment management policies and practices. 

Student Policies on Student Assessment . Student involvement was a three item index that 
emerged in this subsection of assessment management policies and practices for student 
assessment This index measures the extent to which institutions used the following policies or 
practices to promote student involvement in assessment activities: informing students about 
assessment purposes; requiring students to participate in assessment activities; and providing 
students with individual feedback regarding their performance on assessment measures. One item, 
encouraging student participation with incentives, did not load on this factor but was retained as an 
item variable in later data analyses. 

, Professional Development . Factor analysis produced two factors in this subsection. The 
first index, professional development , reflects the extent to which institutions used professional 
development to encourage faculty and academic administrators to support, conduct, or use results 
from student assessment activities. Items loading on this factor included providing faculty with 
funds to attend or present at conferences on student assessment, assistance for using assessment, 
and offering workshops on student assessment for faculty and for academic administrators. A 
second index, student affairs , measures the extent to which institutions required training on student 
assessment for student affairs staff or provided workshops on student assessment for student 
affairs ad minis trators. One item, requiring faculty to receive training on student assessment, did 
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not load on either factor. Compared to other items in this section, requiring training was an 
uncommon practice within institutions. It was retained as an item variable in later data analyses. 

Faculty Evaluation and Rewards. One index emerged in this subsection. Faculty 
evaluation reflects the extent to which institutions’ considered assessment-related criteria in 
evaluation and reward decisions for faculty. Items loading on this factor included whether 
promotion and salary evaluations considered evidence of student performance, whether faculty 
participation in or scholarship on student assessment was considered in promotion, tenure or salary 
reviews, and whether faculty were publicly recognized for their assessment efforts. Two items, 
considering assessment skills in faculty hiring decisions and encouraging faculty to assess student 
learning, did not load on this factor. They were retained as item variables in later data analyses. 

Academic Planning and Review Policies . All four items in this section loaded on a single 
factor. Academic planning and review is an index reflecting the extent to which institutions 
incorporate student performance data into planning or review processes for academic departments 
or programs, general education or core curriculum; courses, and student academic support 
services. 

8.1.4 Factors Related t^Jnstitutional Uses and Impacts of Student Assessment 

Two separate factor analyses of the items in this domain yielded five distinct factors. 

Decision Making . Two factors emerged from the analysis of this survey section: academic 
decisions and faculty decisions. The first, academic decisions, reflects the extent to which student 
assessment information influenced the following academic decisions: creating or modifying 
instructional experiences such as teaching methods, distance learning initiatives, or students’ out- 
of-class learning experiences; developing or revising academic plans or structures such as 
undergraduate academic mission or goals, academic programs or majors, general education 
curriculum, student academic support services, or student assessment plans and processes; 
designing or reorganizing student affairs units; and allocating resources to academic units. Faculty 
decisions measures the extent to which student assessment information influenced institutions* 
decisions regarding faculty promotion and tenure and salary increases or rewards. 



Institutional Impacts. Factor analysis of this section produced three factors related to the 
documented impact of student assessment information on various areas of institutional 
performance: faculty impacts, student impacts and external impacts. The first index, faculty 
impacts, reflects the extent to which student assessment information has had a positive impact on 
faculty satisfaction, discussions of undergraduate education, interest in teaching, and changes in 
teaching methods. Student impacts measures the extent to which student assessment information 
has contributed to student retention or graduation, grade performance, achievement on external 
examinations and satisfaction. Finally, external impacts concerns the extent to which institutions 
have documented positive impacts of student assessment information on several external indicators 
of institutional performance: student applications or acceptance rates, state funding, regional 
accreditation evaluation, private fund-raising results, grant application success, communication 
with external constituents, and institutional reputation or image. 

8.2 Variables Derived by Summing Item Scores 
Several sections of the survey inventory consisted of dichotomous or categorical variables 
which did not lend themselves to factor analysis. In these instances, we reduced data by summing 
scores within a particular section, or in some instances, across two or more related sections. The 
resultant additive index score indicates the number of policies or practices in existence at 
institutions and thus provides an indication of the extensiveness of these policies or practices. A 
detailed description of each index and the survey items associated with ea ch is displayed in Table 
8.2. The indices produced using this procedure are discussed below by survey inventory section. 
8-2.1. Additive Indices Relat ed to Institutional Approach to Student Assessment 

Four sections of this domain of the survey inventory had categorical variables. Five 
additive indices of items with similar content were created. 

Type. Ext ent and Timing of Student Assessment . Two additive indices were created in this 
section. Comprehensiveness of data collection captures the extent to which institutions collected all 
of the types of undergraduate student performance data listed in the survey. This is a summary 
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Tabl e 8.2 Additive Indices by Section of Questionnaire 

Section of Survey Additive IndeX-Labdand-Yadablca Response Scoring Possible Range of 

' Scores 

I. Institutional Approach to 
Student Assessment 
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A. Resource Allocation for - Additive Index 11 - Resource Allocation Practices 
Student Assessment XIVA1 to I VA4 response score 
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measure of institutional responses for all fourteen types of data which were summed to indicate the 
overall comprehensiveness of data collection efforts (1 = not collected; 2 = collected for some 
students; 3 = collected for many students; 4 = collected for all students). Possible index scores 
ranged from 14 to 56. Timing of data collection is an additive index that measures the total number 
of time points at which assessment data were being collected on currently enrolled students. Index 
scores indicating the number of times at which each of nine types of assessment data was collected 
could range from 9 to 45 (1 = not collected; 2 = collected at one point in time; 3 = collected at entry 
and while enrolled or while enrolled and at exit; 4 = collected at entry and at exit; 5 = collected at 
entry, while enrolled and at exit). 

Student Assessment Instruments . An index, number of instruments, was created to 
summarize the number of student assessment instruments or tests being used by institutions. The 
number of instrument sources for each of ten student assessment content areas was summed to 
create an index for this section (1 = yes, 0 = no for each source — institutionally developed, state 
provided, commercially available — for each item). Possible index scores ranged from 0 to 30. 

1- Student Assessment Studies . An index, number of studies, was developed to summarize 
the number of studies institutions had conducted on the relationship between students’ institutional 
experiences and students’ performance. Institutions could report conducting studies on up to nine 
aspects of students’ curricular or co-curricular experiences (0 = no, 1 = yes). Possible index 
scores ranged from 0 to 9. 

Student Performance Profiles or Reports . The additive index, number of reports, measures 
the number of levels at which student assessment data are aggregated and provided as profiles or 
reports of student performance. Institutions could indicate providing reports at none or all of five 
levels of aggregation: institution wide, schools or colleges, acade m ic programs or departments, 
special populations or subgroups of students, course or groups of courses (0 = no, 1 = yes). 
Possible index scores ranged from 0 to 5. 
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8.2.1 Additive Indices Related to Organizational and Administrative Support for Student 
Assessment 

Institutio nal Emphasis . The index, mission emphasis, reflects the extent to which 
institutions’ mission statements explicitly emphasized undergraduate student perfo rman ce, and its 
assessment. Institutions reported whether or not their mission statement explicitly emphasized 
excellence in undergraduate education, identified intended educational outcomes for students, and 
referred to student assessment as an important institutional activity (0 = no, 1 = yes). Possible 
index scores ranged from 0 to 3. 

Administrative and Governance Activities . The index, administrative and governance 
activities, counts the number of administrative or governance activities used by ins titutions to 
promote student assessment Institutions reported whether they had implemented any of the 
following administrative activities: annual institution-wide initiatives on student asse ssm ent, 
rewards or incentives to administrators who promoted student assessment in their uni ts, use of 
student assessment information in evaluation or improvement efforts, or assessment workshops 
for administrators. Further, they reported whether the board of trustees, faculty or students were 
represented in assessment governance (0 = no, 1 = yes). Possible index scores ranged from 0 to 
7. 

Su pport for Student Assessment . An index, administrative and faculty support, was 
created to reflect the extent to which institutions’ chief executive officer, academic affairs 
administrators, student affairs administrators and faculty governance representatives support 
undergraduate student assessment activities. Support scores for each of these four constituencies 
(1 = very unsupportive, 2 = somewhat unsupportive, 3 = neutral or unknown, 4 = somewhat 
supportive, 5 = very supportive) were summed. Possible index scores ranged from 4 to 20. 

Planning and Coordinating Student Assessment Two summative indices were developed 
in this survey section to reflect the breadth of internal representation involved in developing and 
changing assessment plans and policies. Breadth of assessment planning group sums the number 
of four functional areas and five internal constituent groups represented on institutions’ committees 
or groups for student assessment planning and policy setting. Possible index scores ranged from 0 



(meaning no such group or committee existed) to 9. Number approving changes is an index that 
summarizes the number of five functional areas or six internal constituent groups involved in 
approving changes in institutions’ plans or policies for student assessment (0 = no, 1 = yes). 
Possible index scores ranged from 1 to 1 1 . 

8.2.2 Additive Indices Related to Assessment Management Policies and Practices 

Four sections of the domain of Assessment Management Policies and Practices had 
categorical variables. Seven additive indices were created to represent these sections. 

Resource Allocation for Student Assessment . Two summative indices were created to 
measure the nature of resource allocation policies or practices developed by institutions to support 
the collection and use of student assessment information. Resource allocation practices is the more 
inclusive of the two. This index sums the total number of resource allocation practices reported by 
institutions. It includes having an explicit internal budget allocation for student assessment, 
informally considering student assessment information in resource allocation to academic units, 
using student assessment information to competitively allocate resources to academic units, and 
rewarding academic units for improvement based on past student performance indicators (0 = no, 1 
= yes). Possible index scores ranged from 0 to 4. The second index, budget decisions, reflects 
whether institutions had formally used student assessment information in the budget process, either 
to competitively allocate resources among academic units or to reward units for improvements 
relative to past student performance indicators (0 = no, 1 = yes). Possible index scores ranged 
from 0 to 2. 

Student Assessment Inform ation System. The index, computer support , reflects the 
capacity of institutions to collect and manage student assessment information. Institutions reported 
whether they had student information systems that included student performance indicators, could 
track individual students, and had student data integrated with other institutional databases (0 = no, 
1 = yes). Possible index scores ranged from 0 to 3. 

Access to Individual Student Assessmen t Information. An index, access to information, 

t c 

was created to measure the breadth of internal accessibility of assessment information on individual 



students. Institutions reported whether such information was available to institutional researchers, 
senior academic administrators, department chairs or program administrators, student affairs 
professionals or faculty advisors (0 = no, 1 = yes). Possible index scores ranged from 0 to 5. 

Distribution of Student Assessment Reports and Studies . The index, distribution of 
reports , counts the number of constituent groups to whom student assessment reports were 
regularly distributed. Potential recipients included students, faculty, academic administrators, 
student affairs professionals, employers and the general public (0 = no, 1 = yes). Possible index 
scores ranged from 0 to 6. 



8.3 Summary of Data Reduction 

This data reduction has reduced the number of item variables in the survey inventory from 
256 item variables to 33 indices. Thirteen items which did not load on any factor but were deemed 
important were also retained. Table 8.2 identifies these consolidated variables and retrieved i tems 
organized by the domains in our conceptual framework. These indices and individual items 
constitute the variables for our relational analyses in chapters nine, ten and eleven. 
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9. Relationship of External Influences to Institutional Student Assessment 

Patterns 

In chapter four, we reported institutions’ perceptions of state and regional accreditation 
influences on student assessment. In this chapter, we use data from the NCPI research project on 
State Policies and Regional Accreditation Practices of Assessment for Student Learning (Cole et 
al., 1997) regarding state governance structures, state-level approaches to student assessment and 
information concerning regional accrediting practices to more objectively examine the relationship 
of state and regional accreditation activities to institutions’ student assessment patterns (research 
question six). We begin by considering state influences, first presenting descriptive information 
concerning selected dimensions of state governance and state approaches to student agreem ent 
We then examine how these dimensions are related to four institutional domains: (1) approaches to 
student assessment; (2) organizational and administrative support for student assessment; (3) 
assessment management policies and practices and (4) u tilizati on and impacts of student 
assessment Lastly, we examine the relationship between institutions’ regional accreditation 
affiliation and these four institutional domains. 

9. 1 State Approaches to Student Assessment 
States vary widely in their approaches to student assessment (Aper, 1993; Aper et al., 

1990; Boyer et al., 1987; Ewell, 1993). On the basis of our literature review, we identified several 
conceptual dimensions of state-level assessment approaches that may influence institutions’ student 
assessment efforts. Project 5. 1 of NCPI analyzed documents provided by state officials to discern 
characteristics and patterns of state-level assessment approaches (Cole et al., 1997). Into our 
survey database, we merged data from Project 5.1 regarding state governance structures for higher 
education and two dimensions of state approaches to student assessment: form of student 
assessment initiative, and standardization of student assessment indicators and outcomes. Table 
9.1 displays the distribution of these student assessment approach dimensions across states. 
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Table 9.1 Number and Percentage of States' with: 



A. Differing Governance Structures for Higher Education 



Consolidated 


Coordinating 


Coordinating 


Planning 


Total 


Governing 


Regulatory 


Advisory 


Agency 






M 2k 


a % 


M 2k 


M 2k 


is 


2k 


20 39% 


21 41% 


4 8% 


6 12% 


51 


100 


B. Differing 


Initiatives for Student Assessment 3 






Combination of State 


State No State Plan 


Total 


Policy ft Statute statute 


Policy 








U & 


E 2k 


M 2k 


M 2k 


is 


2> 


8 17% 


13 28% 


21 46% 


4 9% 


46 


100 


C Common 


Institutional 


Indicator and 


Outcomes 






Requirements 3 










Common 


Common for 


Institutional No Indicators or 


Total 


for All 


Some 


Specific 


Outcomes 






£ 2k 


E 2> 


IS 2k 


IS & 


E 


& 


15 34% 


8 18% 


12 27% 


9 20% 


44 


100% 



Includes District of Columbia 

Five states did not provide information on this question. 
Seven states did not provide information on this question. 



McGuinness, Epper and Arredondo (1994) developed a continuum of state governance 
structures for higher education, arrayed here in descending order of authority: consolidated 
governing boards; coordinating boards with regulatory authority; coordinating boards with 
advisory capacity; and planning agencies. As Table 9.1-A reveals, states are far more likely to 
have governance structures with greater rather than lesser degrees of authority over higher 
education matters. Eighty percent of states have either a consolidated governing board or a 
coordinating board with regulatory authority. Considerably fewer have planning agencies (12%) 
or coordinating boards with advisory capacity (8%). 

State-level student assessment initiatives (see Table 9.1-B) may take the form of a statute 
established by the state legislature, a policy developed by a state higher education governing board 
or planning agency, or a combination of legislative statutes and non-legislative policies. In close to 
half of respondent states (46%), student assessment initiatives were in the form of policies 

BEST copy available 

o S'- ' 
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developed by state-level higher education officials. Slightly more than a quarter of states (28%) 
have statutes concerning student assessment. A combination of policy and statute has been used 
by 17% of states. Four states (9%) reported they did not have a state plan for student assessment. 

States vary in the extent to which they have standardized the student performance indicators 
or outcomes institutions must report (see Table 9.1-C). States may require the use of a common 
slate of student performance indicators or outcomes for all institutions, may permit institutions to 
devise and report on institutionally-specific indicators or outcomes, or may use a combination of 
state-selected and institutionally-selected indicators or outcomes for some or all of their institutions. 
Of the forty-four states who responded to this question, one-third (34%) require the reporting of 
common student performance indicators or outcomes by all institutions. Approximately one- 
quarter (27%) of states permit institutionally-devised indicators or outcomes. Twenty percent of 
institutions either did not have a student assessment initiative or did not require institutions to 
report any student performance indicators and outcomes. Finally, 18% of institutions required 
some institutions (generally, a specific type of institution) to report common student performance 
indicators or outcomes. 

To examine patterns of state approaches to student assessment, we analyzed the 
relationships among these three state level dimensions. For these analyses, we restricted our 
sample to public institutions. 

Form of Assessment Initiative bv State G overnance Structure. Table 9.2 displays the 
relationship between state governing structures for higher education and form of student 
assessment initiative. Overall, public institution respondents were most likely to be located in 
states with a state-level initiative for student assessment in the form of a policy (38.9%). To a 
lesser extent, institutions experienced initiatives in the form of a statute (31.5%) ora combination 
of st at u te and policy (213%). Less than ten percent (8.4%) of public institutions reported no 
state-level student assessment initiative in place. 
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Table 9.2 Percentage of Public Institutions with Assessment Initiatives by State Governance 
Structure 



Initiative for Student Assessment 


All Inst. 
N=682 


State Governance Structure (46 states)* 


Consolidated Coordinating 
Governing Regulatory 

N=205 N=351 


Coordinating 

Advisory 

N=81 


Planning Chi- 
Agency Square 

N=45 


1. Combination of statute & policy 


21.3 


7.8 


34.8 


8.6 


555.51** 


2. Statute 


31.5 


32.2 


26.8 


67.9 


- 


3. Policy 


38.9 


60.0 


38.5 


- 


15.6 


4. No state plan for assessment 


8.4 


- 


- 


23.5 


84.4 



** p < .01 

"Five states did not provide information on their assessment plan. 



However, there were statistically significant differences in the form of student assessment 
initiative in existence by type of state governing structure for higher education. All institutions in 
states with consolidated governing boards or coordinating regulatory boards were subject to some 
form of student assessment initiative. Institutions in states with consolidated governing boards 
were most likely to have state assessment initiatives in the form of a policy (60%) and to a lesser 
extent, in the form of a statute (32.2%). Less than ten percent (7.8%) had initiatives that combined 
statute and policy. Institutions in states with coordinating regulatory boards for higher education 
were almost equally likely to have state initiatives in the form of a policy (38.5%) or combination 
of statute and policy (34.8%); approximately one-quarter (26.8%) had a state-level statute related to 
student assessment States with coordinating advisory boards were the most likely of all types of 
governance structures to legislate student assessment requirements (67.9% of institutions). Almost 
one-quarter (23.5%) of institutions in states with coordinating advisory boards had no state 
initiative for student assessment and less than ten percent (8.6%) had a state initiative that 
combined statute and policy. Finally, institutions in states with a planning agency for higher 
education were most likely of all governance structures to have no state initiative for student 
assessment (84.4%). When such initiatives did exist, they were in the form of a policy (15.6%). 

Common Indicators/Outcomes bv State Governance Structure . Table 93 displays the 
relationship between state governing structures and their emphasis on common student indicators. 
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Table 9.3 Percentage of Public Institutions with Common Indicators/Outcomes by State 
Governance Structure 



Indicators & Outcomes 


All Inst. 
N=750 


State Governance Structure (44 states) 1 


Consolidated 

Governing 

N=243 


Coordinating 

Regulatory 

N=344 


Coordinating 

Advisory 

N=110 


Planning 

Agency 

N=53 


Chi- 

Square 


1. 


Common for all 


31.5 


10.7 


42.4 


56.4 


17.0 


362.59** 


2. 


Common for some 


26.6 


42.4 


20.9 


20.0 


- 




3. 


Institutional specific 


27.8 


35.4 


33.7 


- 


3.8 




4. 


No indicators or outcomes 


14.1 


11.5 


2.9 


23.6 


79.2 





**p < .01 

•Seven states did not provide information on their indicators or outcomes. 



Slightly less than one-third (31.5%) of public institutions had state-level assessment 
initiatives mandating common student performance indicators or outcomes for all institutions. 
Approximately one-quarter of institutions were in states requiring the reporting of institutionally- 
specific indicators or outcomes (27.8%) or some common indicators or outcomes (26.6%). 
Fourteen percent of institutions were in states that did not have a student assessment initiative or 
did not require the reporting of student performance indicators or outcomes. 

There were statistically significant differences in the standardization of assessment 
indicators and outcomes by the form of state governing structure for higher education. Compared 
to institutions in states with other types of governing structures, institutions with consolidated 
governing boards were most likely to have to report some state-mandated common indicators or 
outcomes (42.4%) or institutionally-specific indicators or outcomes (35.4%), and were least likely 
to have indicators or outcomes common to all institutions (10.7%). 

Institutions in states with coordinating regulatory boards were least likely to have no state 
initiative for student assessment or no requirements for reporting student performance indicators or 
outcomes (2.9%). They were almost as likely as institutions in states with consolidated governing 
boards to report institutionally-specific indicators or outcomes (33.7%) and were second highest in 
having indicators or outcomes common to all (42.4%) or some (20.9%) institutions. More than 
half of institutions (56.4%) in states with coordinating advisory boards had to report indicators or 
outcomes that were common to all state institutions; none reported institutionally-specific indicators 
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or outcomes. Almost one-quarter (23.6%) either had no state-level assessment initiative or 
requirements to report indicators or outcomes. More than three-quarters (79.2%) of institutions in 
states with planning agencies for higher education had no state requirement to report indicators or 
outcomes. When such requirements existed, they were more likely to be common across all 
institutions (17.0%) than common for some (0%) or institutionally-specific (3.8%). 

Common Indicators/Outcomes bv Form of Assessment Initiative . Table 9.4 displays the 
relationship between the form of state assessment initiative and their use of common indicators. 



Table 9.4 Percentage of Public Institutions with Common Indicators & Outcomes by State 
Initiative for Student Assessment 









State Initiative for Student Assessment (44 states)* 








Policy & 


State 


State 


No State 


Chi- 




Indicators & Outcomes 


All Inst. 


Statute 


Statute 


Policy 


Plan 


Square 






N=609 


N=103 


N=203 


N=246 


N=57 


1 . 


Common for all 


31.5 


30.1 


54.7 


20.3 


— . 


56831** 


2. 


Common for some 


26.6 


42.7 


41.4 


13.8 


— 




3. 


Institutional specific 


27.8 


27.2 


- 


56.5 


3.5 




4. 


No indicators or outcomes 


14.1 


- 


3.9 


9.3 


96.5 





** p < .01 

*Seven states did not provide information on their indicators or outcomes. 



There were statistically significant differences in the standardization of assessment 
indicators and outcomes and the form of state initiative for student assessment States u tilizin g a 
combination of policy and statute in their assessment initiative always had some form of 
requirement for reporting indicators or outcomes! They were most likely to require some common 
indicators or outcomes (42.7%) and made moderately high use of indicators/outcomes that were 
common for all institutions (30.1%) and institutionally-specific ( 212 %). States using statutes to 
embody assessment requirements were most likely of all forms of initiatives to have common 
indicators for all institutions (54.7%) , made moderately high use of some common indicators 
(41 .4%) and did not permit institutions to select and report their own indicators and outcomes. 
Conversely, states with policies for student assessment were most likely to permit the reporting of 
institutionally-specific indicators and outcomes (56.5%) and made little use of common indicators 
(13.8% common for some; 20.3% common for all). As would be expected, virtually all states 
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( 96 . 5 %) without any form of state initiative for assessment had no requirements for reporting 
student performance indicators or outcomes. 

Summary . Clear differences emerged in the occurrence and form of state-level student 
assessment initiatives by type of state governance structure for higher education. There was a 
positive relationship between the authority invested in the higher education governance structure 
and the likelihood of having a state-level initiative for student assessment. States with coordinating 
regulatory boards had a mixed array of assessment initiatives, while those with consolidated 
governing boards relied heavily on policies, those with coordinating advisory boards most often 
used legislative means, and those with planning agencies generally had no initiative in place. 

Similarly, there was a positive relationship between having to report student performance 
indicators or outcomes, whether state-mandated or institutionally-devised, and the authority of the 
higher education governance structure. States with consolidated governing boards were more 
likely than those with other governance structures to mandate some common indicators or 
outcomes or to permit institutionally-specific indicators or outcomes to be reported. States with 
coordinating advisory boards made the greatest use of common indicators for all institutions. 

Again, states with coordinating regulatory boards were more varied in their reporting requirements 
and those with planning agencies seldom required any indicators or outcomes from institutions. 

States usin g purely legislative means (statute only) to direct student.assessment initiatives 
mandat^ common performance indicators or outcomes for all or some institutions. Those with 
state-level policies more often permitted institutionally-specific indicators or outcomes. States 
using a combination of policy and statute fell in the middle range of having standardized or 
institutionally-specific assessment indicators or outcomes. 

Finall y some patterns across all three dimensions of state assessment approaches are 
apparent States utilizing a consolidated governing board for higher education were most likely to 
have policies regarding student assessment and to permit institutionally-specific indicators or 
outcomes to be reported. States with coordinating regulatory boards showed two main patterns, 
either having a combination of statute and policy and requiring common indicators or outcomes for 



some or all institutions, or establishing student assessment policies and permitting institutionally- 
specific indicators or outcomes. States with coordinating advisory boards for higher education 
most often enacted statutes regarding student assessment and mandated common indicators or 
outcomes for all institutions. States with planning agencies for higher education were unlikely to 
have any form of initiative or indicators/outcomes for student assessment in postsecondary 
institutions. 

9.2 State Assessment Approaches and Institutional Approaches to Student Assessment 

We now examine how the three state assessment dimensions discussed above (governance 
structure for higher education, form of assessment initiative, standardization of indicators and 
outcomes) relate to the following dimensions of institutions’ student assessment approaches: 
extent of student assessment data collected; student assessment data collection methods; and 
student assessment studies and reports produced. The tables in this section show mean scores for 
each institutional assessment approach dimension. ANOVAs were used to identify statistically 
significant differences among dimensions of state assessment approaches. 

9.2.1 State Assessment Approaches and Extent of Student Assessment 

The extent of institutions’ data collection efforts for student assessment was represented by 
scores on three factors (cognitive assessment, affective assessment and postcollege assessment), 
three single variables that did not load on these factors (academic intentions, academic progress and 
civic/social roles of former students) and two additive indices (comprehensiveness of data 
collection and timing of data collection). 

Extent of Student Assessment bv State Governance Structure . Mean scores and standard 
deviations for each of these extent dimensions for all public institutions by state governance 
structure for higher education are displayed in Table 9.5. The mean scores for all institutions 
restate the profile of student assessment data collection reported in chapter three (see Tables 3.1 
and 3.2). Institutions most often collected information regarding current students’ academic 
intentions (3.22) or academic progress (3.70) and least often collected information regarding 
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Table 9.5 Extent of Student Assessment in Public Institutions by State Governance 



Extent of Student Assessment 
Data Collection: 

Type, Comprehensiveness 
and Timing 


All 

Institutions 

N=885 


State Governance Structure (51 states including DC) 


Consolidated 

Governing 

N=255 


Coordinating 

Regulatory 

N=467 


Coordinating 

Advisory 

N=110 


Planning 

Agency 

N=53 


F 


1. Academic intentions 


3.22 


3.22 


3.18 


3.24 


3.43 


1.08 




(.97) 


(.97) 


(.99) 


(.96) 


(.84) 




2. Academic progress 


3.70 


3.62 


3.72 


3.74 


3.80 


2.20 




(.59) 


(.62) 


(.58) 


(.58) 


(.46) 




3. Cognitive assessment 


1.62 


1.63 


1.69 


1.27 


1.68 


16.86** 




(.57) 


(.55) 


(.57) 


(.52) 


(54) 




4. Affective assessment 


1.74 


1.79 


1.75 


1.58 


1.69 


4.92** 




(.49) 


(.49) 


(JO) 


(.46) 


(.43) 




5. Civic or social roles 


1.55 


1.47 


1.62 


1.44 


1.53 


2.90* 




(.78) 


(.69) 


(.85) 


(.65) 


(.75) 




6. Postcollege assessment 


2.27 


2.27 


2.35 


1.96 


2.28 


13.70** 




(.59) 


(.56) 


(.59) 


(.62) 


(.45) 




7. Comprehensiveness of data 


35 


35 


36 


32 


36 


1327 ** 


collection 


(7) 


(7) 


(7) 


(6) 


(6) 




8. Timing of data collection 


18 


19 


19 


17 


19 


5.17** 




(4) 


(4) 


(4) 


(4) 


(4) 





*p < .05; ** p < .01 
Note: Standard deviations are in parentheses, 
estimated using one-way ANOVA. 



Differences across group means for governance structure were 



affective competencies (1.74), cognitive competencies such as higher order skills (1.62) and the 
civic/social roles held by former students (1.55). On average, institutions collected dat a at two 
time points during students’ enrollment. 

There were statistically significant differences among state governance structures for six of 
the eight student assessment measures, although three were of little practical significance. Overall, 
institutions in states with coordinating regulatory boards collected the most extensive assess men t 



data, ranking highest on five of six measures for which there were significant differences by 
governance structure. These institutions collected the most extensive information on cognitive 
(1.69) and postcollege competencies (2.35), including social/civic roles (1.62); were second 
highest in collecting data on affective competencies (1.75); and had the highest index scores for 
comprehensiveness (36) and timing (19) of data collection. In contrast, institutions in states with 
coordinating advisory boards had the least extensive data collection efforts. They ranked lowest on 
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all six measures for which there were significant differences. Institutions in states with 
consolidated governing boards and planning agencies fell between these two extremes of 
extensiveness of data collection; their scores more closely approximated those associated with 
coordinating regulatory boards than coordinating advisory boards. 

Extent of Student Assessment bv Form of Assessment Initiative . Table 9.6 displays mean 
scores and standard deviations for the extent of public institutions’ data collection efforts by form 
of state initiative for student assessment. 



Table 9.6 Extent of Student Assessment in Public Institutions by State Initiative for 
Student Assessment 



Extent of Student Assessment 






State Initiative for Student Assessment 




Data Collection: 






(46 States Including DC) 






Type, Comprehensiveness 


All 


Policy 


State 


State No State 




and Timing 


Institutions 


& Statute 


Statute 


Policy 


Plan 


F 




N=682 


N=144 


N=215 


N=265 


N=57 




1. Academic intentions 


3.24 


3.27 


336 


3.10 


3.39 


3.66* 




(.95) 


(.89) 


(.92) 


(.99) 


(.90) 




2. Academic progress 


3.70 


3.68 


3.76 


3.67 


3.75 


1.13 




(-58) 


(-59) 


(57) 


(59) 


(56) 




3. Cognitive assessment 


1.64 


1.74 


136 


1.65 


1.60 


3.07* 




(.56) 


(53) 


(58) 


(56) 


(57) 




4. Affective assessment 


1.75 


1.76 


1.81 


1.71 


1.64 


2.41 




(.49) 


(51) 


(50) 


(.47) 


(.40) 




S. Civic or social roles 


1.54 


1.61 


131 


131 


1.54 ' 


035 




(.78) 


(.86) 


(.80) 


(.73) 


(.73) 




6. Postcollege assessment 


2.28 


2.33 


221 


221 


2.27 


031 




(.58) 


(57). 


(.65) 


(55) 


(.48) 




7. Comprehensiveness of data 


35 


36 


35 


35 


35 


1.66 


collection 


(7) 


(6) 


(7) 


(7) 


(5) 




8. Tuning of data collection 


19 


19 


18 


19 


18 


1.73 




(4) 


(4) 


(4) 


(5) 


(4) 





* p < .05 

Note: Standard deviations are in parentheses. Differences across group means for state initiative were 
estimated using one-way ANOVA. 



There were only two statistically significant differences in mean scores for data collection 
by form of student assessment initiative, and these were of relatively small magnitude. Institutions 
differed in the extent to which they collected data regarding student academic intentions. 
Institutions with no state-level initiative for student assessment collected the most extensive data of 
this kind (3.39), followed by those in states with assessment statutes (3.36), a combination of 
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policy and statute (3.27) and finally, those with student assessment policies (3.10). Mean scores 
for collecting data on students’ cognitive competencies were low for all forms of student 
assessment initiatives. In relative terms, institutions in states with a combination of policy and 
statute collected the most extensive cognitive data (1.74), followed by those with policies only 
(1.65), with no state-level initiative (1.60) and with statute only (1.56). 

Extent of St udent Assessment by Common Indicators/Outcomes . Table 9.7 displays mean 
scores and standard deviations for the extent of public institutions’ data collection efforts by state 
reporting requirements. 



Table 9.7 Extent of Student Assessment in Public Institutions by State Requirement for 
Common Indicators and Outcomes 



Extent of Student Assessment 
Data Collection: 

Type, Comprehensiveness 
and Timing 


All 

Institutions 

N=750 


State Indicators and Outcomes Requirement 
(44 States Including DC) 


Common 
for All 
N=243 


Common 
for Some 
N=197 


Institution 

Specific 

N=204 


No Indicators 
or Outcomes 
N=106 


F 


1. Academic intentions 


3.24 


3.11 


3.28 


332 


330 


238 




(.95) 


(1.02) 


(.91) 


(.89) 


(.94) 




2. Academic progress 


3.70 


3.81 


3.61 


3.66 


3.70 


4.10** 




(.58) 


(.48) 


(.66) 


(.60) 


(.60) 




3. Cognitive assessment 


1.62 


1.51 


1.63 


1.72 


1.64 


5.09** 




(-56) 


(.60). 


(.53) 


(.53) 


(.55) 




4. Affective assessment 


1.75 


1.71 


1.75 


1.80 


1.73 


1.06 




(.49) 


(.51) 


(.49) 


(.48) 


(.46) 




5. Civic or social roles 


1.56 


1.64 


1.47 


133 


1.61 


1.71 




(.79) 


(.88) 


(.70) 


(.79) 


(.70) 




6. Postcollege assessment 


2.28 


2.22 


2.26 


2.34 


2.30 


1.41 




(.60) 


(.65) 


(-56) 


(.60) 


(30) 




7. Comprehensiveness of data 


35 


34 


35 • 


36 


36 


1.43 


collection 


( 7) 


(7) 


(7) 


(7) 


(6) 




8. Tuning of data collection 


19 


18 


19 


19 


18 


233 




(4) 


(4) 


(4) 


(5) 


(5) 





**p < .01 

Note: Standard deviations are in parentheses. Differences across group means for state requirements were estimated 
using one-way ANOVA. 



There were only two statistically significant differences in the extensiveness of data 
collection by state requirements for reporting student performance indicators or outcomes. These 
concerned the collection of data on students’ academic progress and cognitive competencies. 
Institutions in states with common indicators or outcomes for all institutions collected the most 
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extensive data on students’ academic progress (3.81). They were followed by institutions with no 
specified indicators or outcomes (3.70), those permitted to report institutionally-specific 
indicators/outcomes (3.66) and those required to report some common indicators/outcomes (3.61). 
This pattern was somewhat reversed for the extent of collecting data on students’ cognitive 
competencies. Institutions permitted to report institutionally-specific indicators/outcomes were 
most likely to collect cognitive data (1 .72); institutions with no indicators/outcomes (1 .64) and 
those with some common indicators (1 .63) fell in the mid-range, while institutions in states with 
common indicators/outcomes for all institutions were least likely to collect this data (1.51). 

Summary. Coordinating regulatory boards were associated with the most extensive student 
assessment data collection efforts among institutions and coor dinatin g advisory boards with the 
least extensive efforts. Fewer but interesting differences in patterns of data collection were evident 
among forms of assessment initiatives and requirements for reporting indicators or outcomes. 
Institutions more often collected data on student intentions when there was no state assessment 
initiative or when a statute was in place. Institutions in states with assessment policies or a 
combination of policy and statute were more likely to collect data concerning students’ cognitive 
performance. Institutions with common indicators/outcomes or no indicators/outcomes were most 
likely to collect academic progress data while those with institutionally-specific indicators or some 
common indicators were most likely to collect cognitive data Data on students* ^d^m ir 
intentions and progress are fairly easy for institutions to collect and for state-level officials to 
compare across institutions while data on students’ cognitive performance are more complex to 
collect and more difficult to use as a basis for inter-institutional comparisons. It is understandable 
that state assessment approaches that are more formalized (based on statutes) and standardized 
(have common indicators for all institutions) would be more likely to include measures of 
intentions and progress, while those that are less formalized (comprised wholly or partially of 
policies) and more institution-centered (permit institutionally-specific indicators) would be more 
likely to encourage the collection of cognitive data. 
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9.2.2 State Assessment Approaches and Student Assessment Data Collection Methods 

Institutions’ methods of collecting student assessment data were represented by scores on 
two factors (student-centered methods and external methods), three variables that did not load on 
these factors (transcript analysis, external examinations and surveys/interviews of withdrawing 
students) and an additive index of the number of assessment instruments used (comprehensive 
tests or examinations from institutional, state or commercial sources). 

Data Collection Meth ods bv State Governance Structure . Mean scores and standard 
deviations for each of these methods for all public institutions by state governance structure for 
higher education are displayed in Table 9.8. 

Table 9.8 Student Assessment Data Collection Methods in Public Ins titutions by State 



Governance Structure for 


ligher Education 






State Governance Structure (51 States Including DC) 




All 


Consolidated Coordinating Coordinating 


Planning 




Data Collection Methods 


Institutions 


Governing 


Regulatory 


Advisory 


Agency 


F 




N=868 


N=252 


N=460 


N=109 


N=53 




1. Number of instruments 


9 


9 


10 


8 


10 


11.20** 




(4) 


(3) 


(4) 


O) 


(4) 




2. Transcript analysis 


2.12 


2.19 


2.10 


1.97 


2.25 


1.30 




(1.10) 


(1.15) 


(1.09) 


(1.03) 


(1.04) 




3. External examinations 


2.06 


2.08 


2.11 


1.82 


2.04 


13.90** 




(.44) 


(.43) 


(.46) 


(.41) 


(.19) 




4. Surveys/interviews of 


2.21 


2.33 


2.16 


2.04 


2.39 


3.91** 


withdrawing students 


(.93) 


(.96) 


.89) 


(.97) 


(.90) 




5. Student-centered methods 


1.30 


1.30 


132 


1.23 


1.33 


2.79* 




(.28) 


(.27) 


(.28) 


(26) 


(-28) 




6. External methods 


2.06 


2.11 


2.07 


1.90 


2.16 


4.17** 




(.58) 


(-59) 


(-58) 


(.49) 


(.60) 





* p < .05; ** p < .01 

Note: Standard deviations are in parentheses. Differences across group means for governance structure were 
estimated using one-way ANOVA. 



In relative terms, institutions made fairly extensive use of comprehensive tests or 
examinations (assessment instruments) as a means of collecting student assessment data and 
limited use of alternative methods of assessment Those alternative methods with the highest mean 
scores (surveys/interviews of withdrawing students, transcript analysis, external examinations and 
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methods) were only used in some institutions or departments within institutions (mean scores just 
above 2.0). 

Statistically significant differences among state governance structures were found for five 
of the six data collection methods considered in this analysis. The most striking profile was that 
associated with having a coordinating advisory board for higher education. Compared to 
institutions with other forms of governance structures, these institutions made the lowest use of 
assessment instruments (8), external examinations (1.82), surveys/interviews of withdrawing 
students (2.04), student-centered methods (1.23) and external methods (1.90). In contrast, 
institutions with planning agencies as their state governance structure had the highest means for 
three of these five methods (2.39 for surveys/interviews with withdrawing students, 133 for 
student-centered methods and 2.16 for external methods) and were tied with coo rdinatin g 
regulatory boards for using the most assessment instruments (10). Institutions with coordinating 
regulatory boards had the highest use of external examinations (2.1 1). This exception aside, 
institutions with coordinating regulatory boards and with consolidated governing boards maH* 
slightly less use of data collection methods than institutions with planning agencies, but greater use 
than institutions with coordinating advisory boards. 

Data Collection Methods bv Form of Assessment Initiative . Mean scores and standard 
deviations for each of these methods for all public institutions by state assessment initiative are 
displayed in Table 9.9. There were two statistically significant differences in mean scores for data 
collection methods used by form of student assessment initiative, and these were relatively small. 
Institutions with no state assessment initiative and with a combination of policy and statute used an 
average of ten assessment instruments while those with initiatives in the form of a policy or statute 
used nine. Institutions with state assessment initiatives comprised of a combination of policies anH 
statutes, policies alone, or with no with state initiative for assessment made comparatively greater 
use (1.35, 1.34 and 1.32 respectively) of student-centered methods than institutions with state 
statutes for assessment (1.27). 
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Table 9.9 Student Assessment Data Collection Methods in Public Institutions by State 
Initiative for Student Assessment 



Data Collection Methods 


All 

Institutions 

N=673 


State Initiative for Student Assessment (46 States Including DC) 


Policy 
& Statute 
N=141 


State 

Statute 

N=215 


State 

Policy 

N=262 


No State 
Plan 
N=57 


F 


1. Number of instruments 


9 


10 


9 


9 


10 


5.23** 




(4) 


(4) 


(3) 


(3) 


(4) 




2. Transcript analysis 


2.13 


2.01 


2.08 


2.24 


2.02 


1.76 




(1.09) 


(1.01) 


(1.09) 


(1.16) 


(.97) 




3. External examinations 


2.07 


2.07 


2.02 


2.11 


2.12 


1.32 




(-58) 


(52) 


(.63) 


(57) 


(53) 




4. Surveys/interviews of 


2.23 


2.15 


2.16 


2.30 


2.43 


2.05** 


withdrawing students 


(.93) 


(.86) 


(.93) 


(.96) 


(.94) 




5. Student-centered methods 


1.32 


1.35 


1.27 


1.34 


1.32 


3.98** 




(.27) 


(.25) 


(.26) 


(.28) 


(-27) 




6. External methods 


2.07 


3.07 


2.02 


2.11 


2.12 


.98 




(58) 


(52) 


(.63) 


(57) 


(53) 





**/><. 01 

Note: Standard deviations are in parentheses. Differences across group means for state initiative woe es timated 
using one-way ANOVA. 



Data Collec tion Methods bv Common Indicators/Outcomes . Mean scores and standard 
deviations for each of these methods for all public institutions by state reporting requirements for 
are displayed in Table 9. 10. There were only two statistically significant differences in the data 
collection methods used by institutions with different state requirements for reporting student 
performance indicators or outcomes. Institutions with no requirements to report indicators or 
outcomes and those permitted to report institutionally-specific indicators/outcomes madt the more 
extensive use of surveys/interviews of withdrawing students (2.50 and 2.23 respectively) and used 
the greatest number of assessment instruments (10). Institutions required to report all or some 
common indicators/outcomes made comparatively less use of surveys/interviews of withdrawing 
students (2.07 and 2.19 respectively) and used one less assessment instrument (9). 

Summary . Having a coordinating advisory board was associated with the lowest mean use 
of student assessment data collection methods. This parallels results reported in the previous 
section, in which this governance structure was related with the least extensive collection of student 
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Table 9.10 Student Assessment Data Collection Methods in Public Institutions by State 
Requirement for Common Indicators and Outcomes 



Data Collection Methods 


All 

Institutions 

N=740 


State Indicators and Outcomes Requirement (44 States Including 

DC) 


Common 
for All 
N=241 


Common 
for Some 
N=193 


Institution 

Specific 

N=202 


No Indicators 
or Outcomes 
N=105 


F 


1. Number of instruments 


9 


9 


9 


10 


10 


3.05* 




(4) 


(4) 


(4) 


(4) 


(3) 




2. Transcript analysis 


2.12 


2.09 


2.04 


2.19 


2.18 


.77 




(1.10) 


(1.09) 


(1.09) 


(1.15) 


(1.07) 




3. External examinations 


2.05 


2.02 


2.07 


2.03 


2.09 


.69 




(.43) 


(.52) 


(.41) 


( 35 ) 


(.42) 




4. Surveys/interviews of 


2.21 


2.07 


2.19 


2.23 


2.50 


5.34** 


withdrawing students 


(.92) 


(.90) 


(.84) 


(.95) 


(.99) 




5. Student-centered methods 


1.30 


1.27 


1.31 


1.31 


1.32 


1.50 




U8) 


(.27) 


( 27 ) 


(.28) 


(.28) 




6. External methods 


2.07 


2.02 


2.04 


2.12 


2.18 


2.41 




(■ 57 ) 


(.57) 


(.58) 


(ST) 


(.53) 





** p < .01 

Note: Standard deviations are in parentheses. Differences across group means for state requirements were estimated 
using one-way ^NOVA. 



data. Somewhat surprisingly, institutions with planning agencies made the greatest use of data 
collection methods, while institutions with consolidated governing boards and coordinating 
regulatory boards fell in the upper mid-range of mean use scores. There was little practical 
difference in the number of comprehensive tests or e xaminati on instruments used among 
institutions with different forms of state assessment initiatives. However, institutions with state 
statutes made the least use of student-centered methods. The use of these more complex 
assessment methods has been encouraged in the assessment literature because they have the 
potential to provide richer information concerning student performance and may contribute more to 
the improvement of teaching and learning practices. However, compared to tests and surveys, 
these methods are more labor intensive and require greater faculty involvement The use of 
statutes, reflective of a more formalized or centralized state assessment approach, appears to be less 
conducive to institutions making use of these student-centered methods. Finally, institutions 
reporting institutionally-specific student indicators or outcomes and those with no requirement to 
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report indicators/outcomes were more likely to conduct surveys and interviews with withdrawing 
students. 

9.2.3 State Assessment Approaches and Assessment Studies and Reports 

Institutions must do more than simply collect student data. In order for data to prove useful 
for informing institutional practices, they must be analyzed and then presented in some form of 
report The nature and extent of assessment studies and reports conducted by institutions were 
represented by scores on two factors (curricular experience studies and co-curricular experience 
studies), a single variable that did not load on these factors (conducts no studies) and two additive 
indices of the number of assessment studies conducted and number of assessment reports written. 

Studies and Reports bv State Go vernance Structure . Mean scores and standard deviations 
of study and report variables for all public institutions by state governance structure for higher 
education are displayed in Table 9.11. 

Table 9.11 Student Assessment Studies and Reports in Public Institutions by State 



Governance Structure for 


Higher Education 






State Governance Structure (51 States Including DC) 




All 


Consolidated Coordinating Coordinating 


Planning 




Studies and Reports 


Institutions 


Governing 


Regulatory 


Advisory 


Agency 


F 




N=872 


N=251 


N=461 


N=108 


N=52 




1. Number of studies 


2 


2 


2 


2 


2 


.183 




(2) 


(2) 


(2) 


(2) 


<2) 




2. Curricular experience studies 


.14 


.14 


.14 


.14 


.10 


.894 




(.18) 


(.17) 


(.18) 


(.19) 


(.16) 




3. Co-curricular experience 


.18 


.17 


.18 


.18 


.21 


.493 


studies 


( 22 ) 


( 22 ) 


(.22) 


(.22) 


(-24) 




4. Conducts no studies 


.40 


.41 


.39 


.38 


.40 


.143 




(.49) 


(.49) 


(.49) 


(.49) 


(-50) 




S. Number of reports 


3 


2 


3 


3 


3 


5.74** 




(1) 


(1) 


0) 


(1) 


(1) 





** p <.01 

Note: Standard deviations are in parentheses. Differences across group means for governance structure were 
estimated using one-way ANOVA. 



Overall, institutions did little in terms of examining the relationship between various aspects 
of students’ institutional experiences and student performance. On average, institutions studied 
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relationships between two aspects of students’ institutional experiences and performance but a 
large proportion of institutions conducted no such studies (40%). 

There was little variation in the studies and reports conducted by institutions in states with 
different governance structures for higher education. The only statistically significant difference 
concerned the number of reports written. Institutions in s tates with consolidated gover nin g boards 
produced one less report (2), on average, than institutions in states with other forms of higher 
education governance (3). 

Studies and Reports bv Form of Assessment Initiative . As Table 9.12 reveals, there was 
little variation in institutional activity in this domain by the form of state assessment initiative. 
Institutions in states with policies concerning student assessment produced one less report (2) than 
institutions in states with other forms of state assessment initiatives (3). 



Table 9.12 Student Assessment Studies and Reports in Public Institutions by State Initiative 
for Student Assessment 







State Initiative for Student Assessment (46 States Including DC) 




All 


Statute 


State 


State 


No State 




Studies and Reports 


Institutions 


& Policy 


Statute 


Policy 


Plan 


F 




N=674 


N=143 


N=213 


N=262 


N=56 




1. Number of studies 


2 


2 


2 


2 


2 


1.48 




(2) 


(2) 


(2) 


(2) 


(2) 


- 


2. Curricular experience studies 


.14 


.16 


.14 


.14 


.10 


1.81 




(.17) 


(.18) 


(.17) 


(.17) 


(.14) 




3. Co-curricular experience 


.19 


J22 


.17 


.18 


.20 


1.31 


studies 


(•22) 


(-24) 


(.22) 


(-22) 


(22) 




4. Conducts no studies 


.38 


.35 


.39 


.40 


.34 


.40 




(.49) 


(.48) 


(.49) 


(.49) 


(.48) 




5. Number of reports 


3 


3 


3 


2 


3 


3.70* 




(1) 


(1) 


(2) 


0) 


(1) 





* p < .05 

Note: Standard deviations are in parentheses. Differences across group ihmw for state initiati ve were egtimateH 
using one-way ANOVA. 



Studies and Reports bv Common Indicators/Outcomes . Table 9.13 displays mean scores 
and standard deviations of study and report variables for all public institutions by state reporting 
requirements for student assessment The only statistically significant difference that emerged 
was, again, related to the number of assessment reports produced. Institutions in states that 
required the reporting of some common student indicators or outcomes produced one less report 
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(2) than institutions in states with common, institutionally-specific, or no required 
indicators/outcomes (3). 



Table 9.13 Student Assessment Studies and Reports in Public Institutions by State 
Requirement for Common Indicators and Outcomes 



Studies and Reports 


All 

Institutions 

N=737 


State Indicators and Outcomes Requirement 
(44 States Including DC) 


Common 
for All 
N=241 


Common 
for Some 
N=193 


Institution 

Specific 

N=199 


No Indicators 
or Outcomes 
N=104 


F 


1. Number of studies 


2 


2 


2 


2 


2 


.45 




(2) 


(2) 


(2) 


(2) 


(2) 




2. Curricular experience studies 


M 


.14 


.14 


.16 


.12 


1.00 




(.18) 


(.18) 


(.18) 


(.19) 


(.16) 




3. Co-curricular experience 


.18 


.19 


.17 


.18 


.18 


.22 


studies 


(.22) 


(.23) 


(■22) 


(.23) 


Ul) 




4. Conducts no studies 


.39 


.39 


.43 


.36 


.39 


.68 




(.49) 


(.49) 


(.50) 


(.48) 


(.49) 




5. Number of reports 


3 


3 


2 


3 


3 


3.12* 




0) 


(1) 


(2) 


0) 


0) 





*p<. 05 

Note: Standard deviations are in parentheses. Differences across group means for state requirements were 
estimated using one-way ANOVA. 



Summary . Overall, “student assessment studies and reports” is a dimension of institutions’ 
student assessment approach that is largely unrelated to state-level assessment approaches. In large 
part, this lack of association is due to the very low frequency with which institutions have 
conducted relational analyses of student assessment data and institutional experiences or have 
produced reports of student assessment results. However, the lack of relationship may also be 
partly attributable to the nature of the analyses and reports referred to in this dimension. These 
studies and reports are most likely to be produced solely for internal consumption and thus would 
be unaffected by external influences. 



9.2.4 Summary of S tate Assessment Approaches and Institutional 
Assessment 






aches to Student 



Compared to the form of state assessment initiative and requirements for common 
indicators and outcomes, state governance structures for higher education accounted for greater 
variation in the extent of student assessment data collected and die use of data collection methods. 




The presence of a coordinating advisory board for higher education was associated with the least 
extensive institutional approach to student assessment. The exclusive use of statutes to frame state 
assessment initiatives and having no state assessment initiative were both associated with less 
extensive collection of data on students’ cognitive competencies, less use of student-centered 
assessment methods and greater use of intentions and academic progress as measures of students’ 
performance. These latter measures are more easily collected and compared among institutions. 
Permitting institutionally-specific indicators/outcomes was positively associated with collecting 
data on cognitive competencies. The opposite was true for institutions in states requiring common 
indicators/outcomes of all institutions. Finally, there was little connection between state 
assessment approaches and internal assessment studies and reports produced by institutions. 

9. 3 State Assessment Approaches and Organizational/Administrative Support for Stu den t 

Assessment 

In the following sections, we examine how the three state assessment-level dimensions 
(governance structure for higher education, form of assessment initiative, standardization of 
indicators and outcomes) relate to two domains of organizational and administrative support for 
student assessment: institutional support strategy for student assessment; and patterns of 
leadership and governance. The tables in this section show mean scores for specific dimensions 
within each of these institutional domains. ANOVAs were used to identify statistically significant 
differences among dimensions of state assessment approaches. 



9.3.1 State Assessment Approaches and Institutional Sup port Strategy for Student Assessment 
We examined the relationship of state assessment approaches to two dimensions of 
institutions’ internal support strategies: the extent to which the mission statement emphasized 
student assessment and institutions’ purposes for conducting student assessmen t (internal 
purposes, accreditation self-study, state requirements). 

Institutional Support Stra tegy bv State Governance Structure . Mean scores and standard 
deviations for each institutional support strategy dimension for all public institutions by state 
governance structure for higher education are displayed in Table 9.14. 
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Table 9.14 Institutional Support Strategy for Student Assessment in Public Institutions by 
State Governance Structure for Higher Education ’ 



Institutional Support Strategy 
for Student Assessment 


All 

Institutions 

N=875 


State Governance Structure (51 States Including DC) 


Consolidated 

Governing 

N=251 


Coordinating Coordinating 
Regulatory Advisory 

N=464 N=109 


Planning 

Agency 

N=53 


F 


1. Mission emphasis 


1.48 


1.47 


1.55 


1.37 


1.17 


3.50* 




(-90) 


(.93) 


(.89) 


(.86) 


(.81) 




2. Conduct for internal 


2.48 


2.53 


2.49 


2.37 


2.43 


2.85* 


purposes 


(.51) 


(.47) 


(-52) 


(.51) 


(.49) 




3. Conduct for accreditation 


3.59 


3.65 


3.58 


3.50 


3.66 


1.67 




(.67) 


(.60) 


(-67) 


(.80) 


(.62) 




4. Conduct for state 


3.29 


3.32 


3.45 


2.84 


2.77 


1833** 




(.95) 


(.95) 


(-83) 


(1.15) 


(1.05) 





*p< .05; **p< .01 

Note: Standard deviations are in parentheses. Differences across group means for governance structure woe 
estimated using one-way ANOVA. 



Institutions gave moderate emphasis to student assessment in their mission statements 
(1.48). Accreditation self-study was the most important purpose reported for student assessment 
(3.59), followed closely by meeting state requirements (3.29) and to a lesser extent, internal 
purposes (2.48). These results for assessment purposes differ somewhat from those reported in 
chapter four (section 4.1.2). In the present analysis, internal purpose was measured with a factor, 
encompassing four components: internal resource allocation, academic program improvement, 
student achievement, and faculty instructional performance. The first of these, resource allocation, 
received the lowest importance ranking as- a single item. Its inclusion in this factor, although 
conceptually sound, depressed the overall score for this construct. 

Institutions, grouped on the basis of their higher education governance structure, differed 
little in the importance they accorded to accreditation as an assessment purpose. There were 
statistically significant differences in these groups on the other three strategy dimensions: mission 
em phasis, internal purposes and state requirements. Differences were most pronounced for the 
impor tan ce rating given to state requirements as a purpose for assessment Importance scores were 
positively associated with governing structure authority. That is, institutions in states with 
coordinating regulatory boards and consolidated governing boards gave higher importance ratings 
to state requirements (3.45 and 3J32) than institutions in states with coordinating advisory boards 
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(2.84) and planning agencies (2.77). Almost the same pattern existed for internal improvement as 
an assessment purpose, although the magnitude of differences was much smaller. Institutions in 
states with consolidated governing boards gave internal purposes the highest importance rating 
(2.53) followed by coordinating regulatory boards (2.49), planning agencies (2.43) and 
coordinating advisory boards (2.37). Similarly, institutions in states with coordinating regulatory 
boards and consolidated governing boards gave greater emphasis to assessment in their mission 
statements (1 .55 and 1.47 respectively) than those with coordinating advisory boards and planning 
agencies ( 1 .37 and 1.17 respectively). 

Institutional Support Strategy bv Form of A s sessment Initiative Table 9.15 displays 
means scores and standard deviations for each institutional support strategy dimension for all 
public institutions by form of state initiative for student assessment 



Table 9.15 Institutional Support Strategy for Student Assessment in Public Institutions by 
State Initiative for Student Assessment 







State Initiative for Student Assessment (46 States Including DC) 


Institutional Support Strategy 


All 


Policy 


State 


State 


No State 




for Student Assessment 


Institutions 


& Statute 


Statute 


Policy 


Plan 


F 




N=675 


N=144 


N=213 


N=261 


N=57 




1. Mission emphasis 


1.51 


1.58 


1.57 


1.47 


1.30 


1.84 




(90) 


(.90) 


(.89) 


(.93) 


(.87) 




2. Internal purposes 


2.48 


2.53 


2.47 


2.48 


2.36 


1.71 




(.49) 


(.47) 


(.50) 


(.50) 


(.48) 




3. Accreditation purposes 


3.59 


3.57 


3.62 


3.57 


3.67 


.48 




(.67) 


(.64) 


(.67) 


(.68) 


(.66) 




4. State purposes 


3.28 


3.39 


3.43 


3.22 


2.75 


8.73** 


** _ ^ ft i 


(.95) 


(.89) 


(.89) 


(.96) 


(1.12) 





** p < .01 

Note. Sta ndar d deviations are in parentheses. Differences across group niwin^ for state initiative were estimated 
using one-way ANOVA. 



The only statistically significant difference observed among institutions with different 
forms of state assessment initiatives concerned the importance rating for requirements as an 
assessment purpose. As may be expected, institutions with legislated forms of state initiatives 
(statute, and combination of statute and policy) identified state reporting requirements as a more 
important purpose for assessment (3.43 and 3.39) than did institutions from states with assessment 
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policies only (3.22). Institutions without any form of state assessment initiative gave state 
requirements the lowest importance score (2.75). 

Institutional Support Strategy bv Common Indicators/Outcomes . Mean scores and 
standard deviations for each institutional support strategy dimension for all public institutions by 
state reporting requirements for assessment are displayed in Table 9. 16. 



Table 9.16 Institutional Support Strategy for Student Assessment in Public Institutions by 
State > ^e2uirements_fo^_Conimon_^ndicatore_^nd_Outcome^^^^^^^_^^^^^^^^_^^^^^_ 



Institutional Support Strategy 
for Student Assessment 


All 

Institutions 

N=742 


Indicators and Outcomes (44 States Including DC) 


Common 
for All 
N=241 


Common for 
Some 
N=194 


Institution 

Specific 

N=203 


No Indicators 
or Outcomes 
N=106 


F 


1. Mission emphasis 


1.48 


1.58 


1.37 


1.49 


1.45 


1.96 




(.90) 


(.92) 


(.86) 


(.91) 


(.92) 




2. Internal purposes 


2.48 


2.44 


2.48 


2.53 


2.46 


1.32 




(30) 


(31) 


(31) 


(.47) 


(.48) 




3. Accreditation purposes 


3.59 


3.49 


3.60 


3.63 


3.71 


3.07* 




(.67) 


(.73) 


(.63) .. 


(.67) 


(39) 




4. State purposes 


3.25 


3.14 


3.37 


3.46 


2.90 


9.97** 




(.97) 


(1.01) 


(.91) 


(.82) 


(1.13) 





*p < .05, ** p < .01 

Note: Standard deviations are in parentheses. Differences across group means for state requirements were estimated 
using one-way ANOVA. 



The importance of state requirements as an assessment purpose varied significantly among 
institutions grouped on the basis of state reporting requirements for student indicators or outcomes. 
Somewhat unexpectedly, institutions permitted to use institutionally-specific indicators/outcomes 
gave the highest importance ratings to state requirements (3.46) followed by institutions in states 
using some common indicators/outcomes (3.37) and, to a lesser degree, those from states 
requiring common indicators/outcomes across all institutions (3.14). As would be expected, 
institutions not required to report any indicators or outcomes gave the lowest importance rating to 
state requirements (2.90). This pattern was reversed somewhat for institutions’ ratings of the 
importance of accreditation review as a purpose for student assessment Here, institutions without 
an y ctatp requirements for indicators/outcomes gave the highest importance rating to accreditation 
requirements (3.71) followed by institutions in states permitting institutionally-specific indicators 
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or outcomes (3.63), those required to report some common indicators/outcomes (3.60) and those 
from states requiring common indicators/outcomes of all institutions (3.14). 

^utptpaiy . A positive relationship existed between the authority of state governance 
structures and the strength of institutions’ assessment support strategy. Institutions in s tate s with 
planning agencies and coordinating advisory boards consistently had the lowest scores on support 
strategy dimensions while those with coordinating regulatoiy and consolidated governing boards 
had the highest scores. Differences were greatest for the importance given to state requirements as 
a purpose for student assessment. Institutions in states using legislative assessment approaches 
identified state requirements as a more important purpose than those from states with policies or no 
form of assessment initiative. There was an inverse relationship between the extent to which states 
mandated indicators/outcomes to be reported by institutions and institutions’ perceptions of state 
requirements as an important purpose for doing student assessment The less institutions were 
required to report state-mandated student performance indicators or outcomes, the greater the 
importance rating they gave to accreditation self-study as an assessment purpose. 

9.3.2 £tate_A$sessment Approaches and Institutional Leadership/Govemance for Student 
Assessment 

We examined the relationship of state assessment approaches to three dimensions of 
institutions’ leadership and governance patterns for student assessment: the institution-wide 
administrative and governance activities used to promote student assessment; degree of 
administrative and faculty support for student assessment; and the structure and process of 
planning and policy setting for assessment 

Institutional Leadership/ Govemance for Assessment bv State Governance Structure . Mean 
scores and standard deviations for each leadership and governance dimensi on for all public 
institutions by state governance structure for higher education are displayed in Table 9.17. 

On average, institutions had introduced two or three (2.33) of seven institution-wide 
activities to promote student assessment. Faculty and administrators were perceived as being 
somewhat to very supportive of student assessment (17.18). Half of public institutions had a 
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formal centralized policy regarding student assessment (a formally adopted plan or policy 
specifying undergraduate student assessment activities of all academic programs or units). Over 
two-thirds (.69) used an institution-wide group to establish assessment policy, with an average of 
four members per group. On average, three institutional positions or groups had approval 
authority for changes in assessment plans. 

Table 9.17 Assessment Leadership and Governance in Public Institutions by State 



Governance Structure for 


ligher Education 






State Governance Structure (51 States Including DC) 




Ail 


Consolidated 


Coordinating Coordinating 


Planning 




Assessment Leadership 


Institutions 


Governing 


Regulatory 


Advisory 


Agency 


F 


and Governance 


N=876 


N=253 


N=462 


N=108 


N=53 




1. Administrative and 


2.33 


2.28 


2.35 


2.37 


2.43 


.30 


governance activities 


1.20) 


(1-21) 


(121) 


(31) 


(.49) 




2. Administrative and faculty 


17.18 


17.30 


17.30 


1632 


17.02 


2.99* 


support 


(234) 


(2.38) 


(2.48) 


(2.85) 


(2.95) 




3. Formal centralized student 


.51 


31 


.57 


.28 


.49 


10.10** 


assessment policy 


(30) 


(30) 


(30) 


(.45) 


(30) 




4. Institution-wide group 


.69 


.68 


.70 


.69 


.68 


.06 


setting policy 


(.46) 


(.47) 


(.46) 


(.46) 


(.47) 




5. Breadth of assessment 


4 


4 


4 


4 


4 


.98 


planning group 


(2) 


(2) 


(2) 


(1) 


(1) 




6. Number approving changes 


3 


3 


3 


3 


2 


1.46 




(1) 


(1) 


(1) 


(2) 


(1) 




7. No student assessment 


n/a 


n/a 


n/a 


n/a 


n/a 


n/a 


policy 










• ' 





* p < .05; ** p < .01 

Note: Standard deviations are in parentheses. Differences across group means for governance structure were 
estimated using one-way ANOVA. 



There were statistically significant differences in two dimensions of assessment leadership 
and governance among institutions with different state governance structures for higher education. 
Institutions in states using a coordinating regulatory board for higher education governance were 
most likely to have a formal centralized institutional policy for student assessment (.57) followed 
by institutions in states with consolidated governing boards (.51) and planning agencies (.49). 
Institutions in states with coordinating advisory boards were much less likely to have this form of 
assessment policy (.28). There were significant, but smaller, differences in reported administrative 
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and faculty support for student assessment Institutions in states with consolidated governing or 
coordinating regulatory boards reported the highest levels of support (17.30); those with state 
planning agencies reported comparatively lower support (17.02) and those with coordinating 
advisory boards reported the least faculty and administradve support (16.52). 

Institutional leadejship/Govemance for Ass essment bv Form of Assessment Initiative. 
Table 9.18 displays mean scores and standard deviations for dimensions of assessment leadership 
and governance in public institutions by state assessment initiative. Only one statistically 
significant difference emerged in leadership/govemance dimensions among institutions with 
different forms of state assessment initiatives. Institutions in states using a combination of policy 
and statute and those in states using policy as an assessment initiative reported a higher degree of 
administrative and faculty support for student assessment (17.66 and 17.32 respectively) than did 
institutions in states using a statute only for assessment (17.09). Institutions with no s tate plan for 
assessment reported the lowest degree of administrative and faculty support (16.71). 



Table 9.18 Assessment Leadership and Governance in Public Institutions by State Initiative 
for Student Assessment 



Assessment Leadership 
and Governance 


All 

Institutions 

N=677 


State Initiative for Student Assessment (46 States Including DC) 


Policy 
& Statute 
N=143 


State 

Statute 

N=214 


State 

Policy 

N=264 


No State 

Plan 

N=56 


F 


L Administrative and 


2.35 


2.28 


2.31 


2.39 


2.48 


.44 


governance activities 


(1.20) 


0-22) 


(129) 


(1.12) 


(1.18) 




2. Administrator and faculty 


1727 


17.66 


17.09 


17.32 


16.71 


2.64* 


support 


(2.40) 


(2.09) 


(2.57) 


( 228 ) 


(2.80) 




3. Formal centralized policy 


.50 


.59 


.45 


.50 


.46 


2.42 




(-50) 


(.49) 


(.50) 


(.50) 


(.50) 




4. Institution-wide planning 


.70 


.67 


.65 


.75 


.67 


221 


group 


(.46) 


(.47) 


(.48) 


(.43) 


(.47) 




5. Breadth of assessment 


4 


4 


4 


4 


4 


1.87 


planning group 


(2) 


(2) 


(2) 


(1) 


(1) 




6. Number approving changes 


3 


3 


3 


2 


2 


2.17 


^ AC 


(1) 


(1) 


(2) 


(1) 


(1) 





*p < .05 

Note: Standard deviations are in parentheses. Differences across group means for state initiative were estimated 
using one-way ANOVA. 
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Institutional Lea dership/Govemance for Assessment bv Common Indicators/Outcomes . 
Mean scores and standard deviations for assessment leadership and governance dimensions in 
public institutions by state reporting requirements for assessment are displayed in Table 9.19. 

Only two statistically significant differences in assessment leadership and governance were evident 
among institutions compared on the basis of state reporting requirements for common student 
performance indicators/outcomes Institutions permitted to report institutionally-specific 
indicators/outcomes were more likely to have a formal centralized assessment policy (.59) than 
institutions with no specified indicators/outcomes (.49) or those required to report some common 
indicators/outcomes (.49). Institutions in states requiring all institutions to report common 
indicators/outcomes were least likely to have a formal centralized assessmen t policy (.42). 
Institutions in states requiring the reporting of some or all common indicators/outcomes utilized 
one more individual to approve changes to the institutional student assessment plan than 
institutions required to report institutionally-specific or no student indicators/outcomes (3 versus 



2 ). 

Table 9.19 Assessment Leadership and Governance in Public Institutions by State 
Requirements for Common Indicators and Outcomes 



Assessment Leadership and 
Governance 


All 

Institutions 
N=742 ' 


State Indicators and Outcomes Requirement 
(44 States Including DC) 


Common 
for All 
: N=241 


Common 
for Some 
N=193 


Institution 

Specific 

N=203 


No Indicators 
or Outcomes 
N=105 


F 


1. Administrative and 


2.36 


2.33 


237 


235 


2.44 


.17 


governance activities 


(1-22) 


(1-22) 


(1.24) 


(131) 


(1.19) 




2. Administrator and faculty 


17.23 


17.06 


17 31 


1737 


1731 


.62 


support 


(230) 


(2.66) 


(235) 


(2.41) 


(237) 




3. Formal centralized policy 


.49 


.42 


.49 


.59 


.49 


3.96** 




(•50) 


(.50) 


(30) 


(.49) 


(30) 




4. Institution-wide planning 


.71 


.68 


.69 


.75 


.71 


.92 


group 


(46) 


(.47) 


(.46) 


(.43) 


(.45) 




5. Breadth of assessment 


4 


4 


4 


4 


4 


2.40 


planning group 


(2) 


(2) 


(1) 


(2) 


(1) 




6. Number approving changes 


3 


3 


3 


2 


2 


2.85* 




(1) 


(2) 


(2) 


(2) 


(1) 





*p < .05, **p < .01 

Note: Standard deviations are in parentheses. Differences across group means for state requirements were estimated 
using one-way ANOVA. 



260 



216 



Summary . There was a positive relationship between the use of authoritative governance 
structures at the state level (consolidated governing and coordinating regulatory boards) and the use 
of formal centralized assessment policies at the institutional level. Findings also suggest the use of 
more authoritative state governance structures for higher education is not antithetical to internal 
institutional support for student assessment. Beyond the form of governance structure used, it 
seems that less directive or standardized state approaches may be more conducive to promoting 
institutional support for student assessment. The use of policies to frame state assessment 
initiatives, alone or in combination with statutes, was more often associated with institutional 
support for assessment than was the exclusive use of statutes or having no form of initiative. 
Similarly, permitting institutions to develop their own indicators/outcomes to report, rather than 
mandating them, was related to institutions establishing formal centralized assessmen t policies. 

9.4 State Assessment Approaches and Assessment Management Policies and Practices 
In this section we examine the relationships among the three state-level dimensions 
(governance structure for higher education, form of assessment initiative, standardization of 
indicators and outcomes) and the assessment management policies and practices used by 
institutions. The following dimensions of assessment management policies and practices were 
considered in these analyses: institutional evaluation of student assessm ent plan; resource 
allocation practices; access to assessment information; distribution of assessment reports; student 
involvement policies; professional development policies; faculty evaluation and reward policies; 
and academic planning and review policies. ANOVAs were used to identify statistically significant 
differences in the use of these policies/practices among dimensions of state assessment approaches. 

Assessment Management Policies and Practices bv State Governance Structure . Table 9.20 
displays the mean scores and standard deviations related to the use of these assessmen t 
management policies and practices for all public institutions by state governance structure. 

Half (.51) of public institution respondents had formally or informally evaluated their 
student assessment plan or approach. Institutions made minimal use of resource allocation 
practices to support student assessment (1.21); as discussed in chapter five, these practices have 



Table 9.20 Assessment Management Policies and Practices in Public Institutions by State 
Governance Structure for Higher Education 



State Governance Structure (51 States Including DC) 



Assessment Management 
Policies and Practices 


All 

Institutions 

N=866 


Consolidated 

Governing 

N=247 


Coordinating Coordinating 
Regulatory Advisory 

N=446 N=109 


Planning 

Agency 

N=52 


F 


1. Conducted evaluation of 


.51 


.49 


.54 


.47 


.48 


1.03 


assessment approach 


(•50) 


(.50) 


(-50) 


(50) 


(50) 




2. Resource allocation practices 


1.21 


1.17 


1.23 


1.17 


1.24 


.79 




(.49) 


(.48) 


(51) 


(.38) 


(.44) 




3. Access to information 


3.51 


3.46 


3.63 


3.20 


3.36 


2.32 




(1.66) 


(1.72) 


(1.61) 


(1.67) 


(1.78) 




4. Distribution of reports 


2.52 


2.51 


2.63 


2.24 


2.13 


3.50* 




(1.43) 


(1.42) 


(1.46) 


(139) 


(1.11) 




5. Student involvement 


2.65 


2.70 


2.70 


2.36 


2.62 


4.48** 


policies 


(.88) 


(.84) 


(.89) 


(.89) 


(.82) 




6. Student incentives 


1.78 


1.68 


1.85 


1.85 


1.63 


1.41 




(1.19) 


(1.17) 


(123) 


(1.19) 


(.93) 




7. Professional development 


1.94 


1.89 


1.98 


1.79 


2.16 


3.12* 


policies 


(.80) 


(.81) 


(.83) 


(.64) 


(.72) 




8. Faculty training required 


2.47 


2.55 


2.52 


2.00 


2.63 


3.76* 




(1-54) 


(155) 


(155) 


(133) 


(1.68) 




9. Student affairs policies 


2.05 


2.15 


2.02 


1.94 


2.08 


.98 




(1.21) 


(1-25) 


(221) 


(1.11) 


(1.19) 




10. Faculty evaluation policies 


1.18 


1.17 


1J22 


1.08 


1.11 


1.83 




(.61) 


(.65) 


(.61) 


(58) 


(59) 




11. Hiring process 


1.70 


1.78 


1.72 


1.42 


1.78 


2.67* 




(1.12) 


(1.17) 


(1.15) 


(.75) 


(1.15) 




12. Encourage faculty to assess 


3.93 


3.90 


4.04 


3.38 


4.19 


8.28** 




(1.30) 


(132) 


; (124) 


(1.41) 


(122) 




13. Academic planning and 


2.80 


2.87 


• 2.85 


2.49 


2.64 


4.83** 


review policies 


(.96) 


(1.00) 


(.96) 


(.89) 


(.80) 





* p < .05; ** p < .01 

Note: Standard deviations are in parentheses. Differences in group means for governance structure were estimated 
using one-way ANOVA. 



more often taken the form of having an explicit budget allocation for assessment than using 
assessment results to make internal resource allocation decisions. Institutions gave quite liberal 
access to assessment information concerning individual students (3.51) but were fairly narrow in 
their range of internal distribution of student assessment reports or studies (2.52). Institutions 
made the most extensive use of policies to incorporate assessment information into academic 
planning and review processes at a variety of levels (2.80). Policies to encourage student 
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involvement in assessment activities were also used quite widely within institutions (2.65) with the 
exception of offering incentives (1.78). Institutions were slightly more likely to provide 
professional development to student affairs staff or administrators (2.05) than to faculty members 
(1.94). Between a few and some departments required faculty to have training in student 
assessment (2.47). Faculty evaluation and reward policies were not commonly used. While 
institutions reported wide use of encouragement for faculty to assess student learning (3.93), few 
departments considered assessment skills when hiring faculty (1.70) and almost none considered 
assessment participation, scholarship or student performance among evaluative criteria for faculty 
(1.18). 

Comparisons of the extent of institutional use of assessment management policies/practices 
on the basis of their state governance structure for higher education revealed statistically significant 
differences in seven dimensions: distribution of assessment reports; student involvement; 
professional development; requiring faculty training in assessment; considering a ss ess m ent skills 
in hiring; encouraging faculty to assess; and academic planning and review. With little exception, 
these differences reflected the significantly less extensive use of policies/practices among 
institutions in states with coordinating advisory boards compared to institutions in states with the 
other three types of governance structures. 

Institutions in states using planning agencies for higher education mad* the most extensive 
use of assessment management policies and practices overall. They scored highest on providing 
professional development in assessment for faculty (2.16), requiring faculty t rainin g in assessment 
(2.63), considering assessment skills when hiring faculty (1.78) and encouraging faculty to assess 
student learning (4.19). They fell in the mid-range of student involvement policies (2.62) and 
using assessment data for academic planning and review (2.64), and had the lowest distribution of 
assessment reports (2.13). 

In comparison to institutions with planning agencies at the state level, institutions with 
consolidated governing boards and coordinating regulatory boards made slightly less use of these 
assessment management policies and practices. Institutions with consolidated governing boards 
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reported the most extensive use of policies to encourage student involvement (2.70), consider 
assessment skills in faculty hiring decisions (1.78) and incorporate assessment data in academic 
planning and review processes (2.87), and were in the upper mid-range of distribution of 
assessment reports (2.70), professional development for faculty (1.89), faculty training (2.55), 
and the encouragement of faculty to assess student learning (3.90). 

Institutions with coordinating regulatory boards had a similarly strong profile of policy use. 
They scored highest on two policies — distributing assessment reports (2.63) and encouraging 
student involvement (2.70) and were neither highest nor lowest in offering professional 
development for faculty (1.98), requiring faculty training in assessment (2.52), considering 
assessment skills when hiring faculty (1.72), encouraging faculty to assess student learning 
(4.04), and using assessment data in academic planning and review processes (2.85). 

Institutions with coordinating advisory boards had the lowest reported use on six of seven 
items for which there were statistically significant differences: student involvement policies (2.36), 
offering professional development for faculty (1.79), requiring faculty tr aining in assessment 
(2.00), considering assessment skills when hiring faculty (1.42), encouraging faculty to assess 
student learning (3.38) and using assessment data in academic planning or review processes 
(2.49). They were second lowest in terms of distributing assessment reports (2.24). 

Assessment Management Policies and Practices bv Fo r m of Assessment Initiative . Table 
9.21 displays mean scores and standard deviations for assessment management policies and 
practices in public institutions by form of state initiative for student assessm e n t 

Statistically significant but small differences were observed in the use of two assessment 
management policies and practices by institutions with different for ms of state assessment 
initiatives: student affairs pblicies and the use of assessnient data in academic planning and review 
processes. In both instances, institutions with no state initiative for assessment had the lowest use 
scores (1.80 and 2.50 respectively). Use scores for states with statutes, policies, and a 
combination of policies and statutes for assessment were comparatively higher and more closely 
clustered together. Institutions with statutes for assessment reported the most extensive provision 
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Table 9.21 Assessment Management Policies and Practices in Public Institutions by State 
Initiative for Student Assessment Plan 



State Initiative for Student Assessment (46 States Including DC) 



Assessment Management 
Policies and Practices 


All 

Institutions 

N=668 


Policy & 
Statute 
N=142 


State 

Statute 

N=213 


State 

Policy 

N=257 


No State 
Plan 
N=56 


F 


1. Conducted evaluation of 


.52 


.54 


.59 


.46 


.50 


2.51 


assessment approach 


(.50) 


(.50) 


(.49) 


(50) 


(50) 




2. Resource allocation practices 


1.21 


1.25 


1.24 


1.16 


1.19 


.93 




(.48) 


(-54) 


(.49) 


(.44) 


(.40) 




3. Access to information 


3.51 


3.42 


3.57 


3.55 


3.29 


.62 




(1.66) 


(1.71) 


(1.66) 


(1.61) 


(1.79) 




4. Distribution of reports 


2.60 


2.65 


2.65 


2.63 


2.20 


1.61 




(1.45) 


(1.43) 


(1.49) 


(1.47) 


(126) 




5. Student involvement 


2.62 


2.68 


2.70 


2.56 


2.49 


1.41 


policies 


(.88) 


(.81) 


(.93) 


(.87) 


(.85) 




6. Student incentives 


1.78 


2.00 


1.80 


1.68 


1.63 


2.52 




(1.17) 


(1.17) 


(122) 


(1.16) 


(.90) 




7. Professional development 


1.96 


2.05 


1.88 


1.96 


2.02 


1.37 


policies 


(.79) 


(.80) 


(.75) 


(.83) 


(.73) 




8. Faculty training required 


2.44 


2.40 


2.36 


2.55 


2.33 


.74 




(1-53) 


(153) 


(1.46) 


(156) 


(159) 




9. Student affairs policies 


2.06 


1.92 


2.24 


2.05 


1.80 


2.95* 




(1-21) 


(1.10) 


(127) 


(124) 


(1.02) 




10. Faculty evaluation policies 


1.19 


1.28 


1.20 


1.15 


1.14 


1.48 




(.62) 


(.63) 


(.65) 


(59) 


(.60) 




11. Hiring process 


1.70 


1.65 


1.66 


1.78 


1.63 


.69 




(1.12) 


(110) 


(1.10) 


(1.17) 


(1.07) 




12. Encourage faculty to assess 


3.97 


4.06 


3.81 


4.07 


3.86 


1.97 




(1.26) 


(1.14) 


(155) 


(122) 


(159) 




13. Academic planning and 


2.78 


2.90 


2.85 


2.73 


2.50 


2.91* 


review policies 


(.94) 


(.89) 


(.93) 


(.99) 


(.86) 





• p < .05 

Note : Standard deviations are in parentheses. Differences across group himik for state initiative were e stimated 
using one-way ANOVA. 



of professional development for student affairs staff and administrators ( 23 . 4 ) followed by 



institutions with state assessment policies (2.05) and those with a co mbinatio n of state policies and 
statutes on assessment (1.92). Institutions in states using a combination of assessment policies 
and statutes reported the greatest use of assessment data in academic planning and review decisions 
(2.90) followed closely by institutions with state statutes (2.85) and institutions with state policies 
(2.73). 
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Assessment Management Policies and Practices hv Common Indicators/Outcomes . Mean 
scores and standard deviations of public institutions’ use of assessment management policies and 
practices by state reporting requirements for assessment are displayed in Table 9.22. 



Table 9.22 Assessment Management Policies and Practices in Public Institutions by State 
Requirement for Common Indicators and Outcomes 



Assessment Management 
Policies and Practices 


All 

Institutions 

N=736 


State Indicators and Outcomes Requirement 
(44 States Including DC) 


Common 
for All 
N=240 


Common for 
Some 
N=195 


Institution 

Specific 

N=199 


No Indicators 
or Outcomes 
N=102 


F 


1. Conducted evaluation of 


.51 


.48 


.53 


.53 


.47 


.86 


assessment approach 


(.50) 


(50) 


(50) 


(50) 


(50) 




2. Resource allocation practices 


1.22 


1.23 


1.22 


1.25 


1.15 


.63 




(.50) 


(50) 


(54) 


(51) 


(36) 




3. Access to information 


3.50 


3.58 


3.43 


337 


3.35 


.70 




(1.66) 


(1.69) 


(1.69) 


(1.60) 


(1.69) 




4. Distribution of reports 


2.54 


2.53 


2.55 


2.65 


2.37 


.90 




(1.40) 


(147) 


(1.40) 


(1.41) 


(121) 




5. Student involvement 


2.64 


2.60 


2.80 


2.62 


2.51 


2.24 


policies 


(.88) 


(.93) 


(.87) 


(.83) 


(.84) 




6. Student incentives 


1.77 


1.73 


1.85 


1.84 


1.55 


1.83 




(1.18) 


(1.16) 


(1.18) 


(132) 


(.88) 




7. Professional development 


1.96 


1.79 


1.98 


2.09 


2.03 


5.55** 


policies 


(.81) 


(.74) 


(.83) 


(.87) 


(.70) 




8. Faculty training required 


2.45 


2.24 


2.48 


239 


2.55 


2.17 




(1-53) 


(1.41) 


(157) 


(158) 


(157) 




9. Student affairs policies 


2.07 


1.99 


221 


2.09 


1.93 


1.60 




(1-21) 


(1.17) 


(1-28) 


(123) 


(1.14) 




10. Faculty evaluation policies 


1.17 


1.23 


1,18 


1.15 


1.08 


1.62 




(59) 


(.60) 


(.61) 


(58) 


(56) 




11. Hiring process 


1.70 


1.54 


1.75 


1.87 


1.67 


332* 




(1.11) 


(.91) 


(1.16) 


(129) 


(1.00) 




12. Encourage faculty to assess 


3.91 


3.73 


3.85 


4.11 


4.05 


330* 




(1.30) 


(134) 


(136) 


(122) 


(121) 




1 3. Academic planning and 


2.79 


2.74 


2.79 


2.89 


2.72 


1.06 


review policies • 


(.94) 


(.93) 


(.96), 


(.97) 


«: (.89) 





*p< .05; **p < .01 

Note: Standard deviations are in parentheses. Differences across group means for state requirements were 
estimated using one-way ANOVA. 
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Three small but statistically significant differences were observed in the use of assessment 
management policies and practices among institutions with different state reporting requirements. 
These concerned providing professional development for faculty, considering assessment skills 
when hiring faculty, and encouraging faculty to assess student learning. In all three cases, 
institutions permitted to report institutionally-specific student indicators and outcomes h ad the 
highest use scores (2.09, 1.87, 4.1 1). Conversely, institutions in states requiring the reporting of 
common indicators/outcomes for all institutions reported the lowest use of these policies/practices 
(1.79, 1.54, 3.73). Compared to these two types of reporting requirements, corresponding use 
scores for institutions in states requiring some common indicators/outcomes and states requiring no 
indicators/outcomes were in the middle. 

Summary. As was discussed in chapter five, institutions have matte co mp aratively little 
use of assessment management policies and practices to support student assessment Policies 
regarding student involvement and academic planning were more frequent than policies regarding 
professional development and faculty evaluation. Statistically significant differences in policy use 
were noted by state governance structure. Having a coordinating advisory board for state higher 
education governance was generally associated with the lowest institutional use of these policies 
while having a planning agency was related to greatest use. There was little significant variation in 
the use of assessment management policies and practices to support assessment among institutions 
with different forms of state initiatives for student assessment Requiring institutions to devise and 
report institutionally-specific student performance indicators or outcomes was more often 

associated with institutional use of assessment management policies and practices for s tuden t 
assessment than was having a state requirement to report common indicators/outcomes. 



9.5 State Assessment Approaches and Institutional Uses and Impacts of Student Assessment 
Finally, we examined the relationships among the three state-level dimensions (governance 
structure for higher education, form of assessment initiative, standardization of indicators and 
outcomes) and the extent to which institutions had used and documented impacts from student 
assessment information. As discussed in chapter eight, factor analysis distinguished two 
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dimensions of institutional decision making that could be influenced by student assessment 
information: academic decisions and faculty decisions. Three dimensions of institutional impact 
were identified: faculty impacts (e.g., satisfaction, interest in teaching, teaching methods); student 
impacts (e.g., retention or graduation, grade performance, satisfaction); and external impacts (e.g., 
external reputation or image, external funding received). ANOVAs were used to identify 
statistically significant differences in assessment uses and impacts by dimensions of state 
assessment approaches. 

Assessment Uses and Impacts bv State Governance Structure . Table 9.23 displays the 
mean scores and standard deviations related to assessment uses and impacts for all public 
institutions by state governance structure. 

Table 9.23 Institutional Uses and Impacts of Student Assessment in Public Institutions by 
State Governance Structure for Higher Education 



Uses and Impacts 


All 

Institutions 

N=827 


State Governance Structure (51 States Including DC) 


Consolidated 

Governing 

N=241 


Coordinating Coordinating 
Regulatory Advisory 

N=438 N=103 


Planning 

Agency 

N=51 


F 


1. Academic decisions 


1.40 


1.40 


1.43 


1.34 


1.36 


1.44 




(-40) 


(.40) 


(.41) 


(38) 


(36) 




2. Faculty decisions 


1.23 


1.24 


1.26 


1.20 


1.04 


2.22 




(-58) 


(.60) 


(59) 


(55) 


(34) 




3. Faculty impacts 


1.54 


1.58 


1.53 


1.44 


1.66 


1.26 




(.75) 


(.77) 


(.74) 


(.74) 


(.75) 




4. Student impacts 


1.64 


1.66 


1.67 


1.57 


1.44 


1.61 




(.80) 


(.81) 


(.82) 


(.74) 


(.73) 




5. External impacts 


1.19 


1.18 


1.24 


1.09 


1.10 


2.80* 




(54) 


(54) 


(55) 


(.48) 


(50) 





* p < .05 

Note: S tandar d deviations are in parentheses. Differences in group means for governance structure were estimated 
using one-way ANOVA. 

As was noted in chapter seven, institutions have reported limited influence of student 
assessment information on institutional decisions and few have monitored the impact of assessment 
information on internal and external performance indicators. Assessment information was more 
likely to have influenced academic decisions (1.40) than faculty decisions (1.23) but both mean 
scores suggest this information was not very influential. Institutions are somewhat more likely to 
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have documented positive impacts from student assessment on internal performance indicators 
(1.54 for faculty impacts and 1.64 for student impacts) than external performance indicators 
(1.19), but again, mean scores reveal little documentation. 

There was little variation in uses and impacts of student assessment by form of s tate 
governance structure for higher education. One small but statistically significant difference was 
found. Institutions in states using coordinating regulatory boards were most likely to have 
documented positive external impacts from student assessment information (1.24), while 
institutions from states with coordinating advisory boards were least likely (1.09). Institutions 
from states with consolidated governing boards (1.18) and with planning agencies (1.10) scored 
between the two other forms of governance structures. 

Assessment Uses and Impacts by Form of Assessment Initiative . Table 9.24 displays 
mean scores and standard deviations of public institutions’ uses and impacts of assessment 
information by state assessment initiative. 



Table 9.24 Institutional Uses and Impacts of Student Assessment in Public Institutions by 
State Initiative for Student Assessment 







State Initiative for Student Assessment (46 States Including DC) 




All 


Policy & 


State 


State 


NoState 




Uses and Impacts 


Institutions 


Statute 


Statute 


Policy 


Plan 


F 




N=642 


N=136 


N=207 


N=250 


N=54 




1. Academic decisions 


1.42 


1.48 


1.43 


1.39 


1.37 


1.74 




(.40) 


(.42) 


(.40) 


(.39) 


. (- 37 > 




2. Faculty decisions 


1.23 


1.28 


126 


1.20 


1.11 


1.40 




(-58) 


(.62) 


(.62) 


(.54) 


(.43) 




3. Faculty impacts 


1.58 


1.63 


1.55 


1.58 


1.49 


.60 




(.75) 


(.74) 


(.74) 


(.77) 


(.73) 




4. Student impacts 


1.67 


1.81 


1.70 


1.64 


1.39 


3.68* 




(.80) 


(.84) 


(.82) 


(.78) 


(.72) 




5. External impacts 


1.20 


1.25 


1.26 


1.17 


.95 


5.49** 




(.54) 


(.52) 


(-59) 


(.52) 


(.39) 





* p < .05; **p < .01 

Note: Standard deviations are in parentheses. Differences across group means for state initiative woe estimated 
using one-way ANOVA. 
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There were few differences in reported uses and impacts of student assessment by the form 
of state initiative for student assessment. Institutions in states using a combination of policies and 
statutes, and those from states using statutes only were statistically more likely to report positive 
impacts of assessment information on students (1.81 and 1.70) and external performance 
indicators (1.25 and 1.26) than institutions with state policies on student assessment (1.64 for 
student impacts and 1.17 for external impacts). Institutions with no state plan were least likely to 
report assessment-related impacts on students (1.39) or the external environment (.95). 

Assessment Uses and Impacts bv Common Indicators/Outcomes . As Table 9.25 reveals, 
there were no statistically significant differences found in the institutional uses and impacts of 
student assessment information reported among public institutions with different state reporting 
requirements for student indicators/outcomes. 



Table 9.25 Institutional Uses and Impacts of Student Assessment in Public Institutions by 
State Requirement for Common Indicators and Outcomes 



Uses and Impacts 


All 

Institutions; 

N=703 


State Indicators and Outcomes Requirement 
(46 States Including DC) 


Common 
r.. for All 
N=228 


Common 
for Some 
N=189 


Institution 

Specific 

N=195 


No Indicators 
or Outcomes 
N=99 


F 


1. Academic decisions 


1.40 


1.38 


1.38 


' 1.42 


1.42 


.52 




(.40) 


(.40) 


(.41) 


(.41) 


(37) 




2. Faculty decisions 


1.22 


1.23 


1.24 


1.19 


1.19 


.48 




(-56) 


(-56) 


(-61) 


(54) 


(53) 




3. Faculty impacts 


1.56 


1.49 


1.55 


1.66 


1.60 


1.88 




(76) 


(.74) 


(.77) 


(-77) 


(.75) 




4. Student impacts 


1.65 


1.64 


1.65 


1.68 


1.61 


.19 




(.80) 


(.79) 


(.84) 


(-80) 


(.77) 




5. External impacts 


1.19 


1.23 


1.20 


1.17 


1.10 


1.42 




(.54) 


(-56) 


(-56) 


(51) 


(51) 





Note: Standard deviations are in parentheses. Differences across group means for state requirements were estimated 
using one-way ANOVA. 



Summary . Few, and generally weak, relationships were found among dimensions of state 
assessment approaches and institutional uses and impacts of student assessment information. In 
large part, this result is due to the low incidence of assessment-related uses and impacts reported 
by institutions. As may be expected, some differences were found in relation to external impacts, a 
construct that included state funding allocation. There was a small positive relationship between 
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the authority of the state governance structure for higher education and the likelihood of institutions 
reporting a positive external impact from student assessment information. Also, institutions in 
which state assessment initiatives were in the form of either statutes or both policies and statutes 
were more likely to report positive external impacts from assessment than those with state policies 
only or no state initiative for student assessment. Interestingly, there were no statistically 
significant differences in documented impacts found among institutions with different types of state 
reporting requirements for student indicators or outcomes. 

9.6 Regional Accreditation Affiliation and Institutional Approach to Student Assessment 
There are six regional accrediting agencies responsible for evaluating institutions of higher 
education within their respective geographical areas: Middle States Association of Colleges and 
Schools, New England Association of Schools and Colleges, North Central Association of 
Colleges and Schools, Northwest Association of Schools and Colleges, Southern Association of 
Colleges and Schools, and Western Association of Schools and Colleges. In this section, we 
present findings regarding relationships among institutions’ regional accreditation affiliation and 
the following dimensions of institutions’ student assessment approach: extent of student 
assessment data collected, student assessment data collection methods, and student assess ment 
studies and reports produced. For these analyses, we include data from public and private 
institutions. ANOVAs were used to identify statistically significant differences in institutional 
assessment approach dimensions by regional accreditation membership. 

9.6.1 Regional Accreditation A ffiliation and Extent of Student Assessment 

The extent of institutions’ data collection efforts for student assessment was represented by 
scores on three factors (cognitive assessment, affective assessment and postcollege assessment), 
three single variables that did not load on these factors (academic intentions, academic progress and 
civic/social roles of former students) and two additive indices (comprehensiveness of data 
collection and timing of data collection). Mean scores and standard deviations for each of these 
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extent dimensions for all institutions by regional accreditation affiliation are displayed in Table 

9.26. 



Extent of Student Assessment 








Accrediting Region 






Data Collection: 


















Type, Comprehensiveness 


All 


Middle 


North 


New 










and Timing 


Institutions 


States 


Central 


England 


Northwest 


Southern 


Western 


F 




N=1393 


N=191 


N=529 


N=87 


N=80 


N=423 


N=83 




1. Academic intentions 


3.25 


3.19 


3.25 


3.14 


3.32 


3.30 


3.21 


.64 . 




(.98) 


(1.03) 


(.96) 


(1.09) 


(.88) 


(.99) 


(1.02) 




2. Academic progress 


3.76 


3.90 


3.70 


3.75 


3.73 


3.77 


3.71 


3.31** 




(.55) 


(.37) 


(59) 


(58) 


(.45) 


(54) 


(.66) 




3. Cognitive assessment 


1.68 


1.54 


1.78 


1.54 


1.59 


1.75 


1.22 


18.23** 




(-58) 


(59) 


(55) 


(.62) 


(58) 


(55) 


(54) 




4. Affective assessment 


1.87 


1.89 


1.84 


1.84 


1.82 


1.94 


1.68 


3.55** 




(54) 


(55) 


(53) 


(59) 


(52) 


(53) 


(57) 




3. Civic or social roles 


1.80 


1.97 


1.77 


1.84 


1.57 


1.82 


1.63 


3.26** 




(.89) 


(1.01) 


(.89) 


(.81) 


(.78) 


(.89) 


(.75) 




6. Post-college assessment 


2.29 


2.41 


2.31 


2.18 


2.18 


2.35 


1.82 


14.22** 




(.60) 


(.65) 


(58) 


(.61) 


(54) 


(58) 


(55) 




7. Comprehensiveness of 


36 


37 


37 


34 


34 


37 


31 


13.68** 


data collection 


(7) 


(7) 


(7) 


C 7) 


(7) 


(7) 


(7) 




8. Timing of data collection 


19 


18 


20 


18 


19 


19 


17 


4.40** 




(5) 


(5) 


(5) 


(4) 


(5) 


(4) 


(5) 





**p < .01 

Note: Standard deviations are in parentheses. Differences across group means for accrediting region were estimated 
using one-way ANOVA. 

Although the inclusion of private institutions in this analysis did result in slightly higher 
mean scores for several measures (affective assessment, civic/social roles of former students. 



timing of data collection) than was the case for public institutions only, the overall pattern of data 
collection presented here replicates the results discussed earlier in this chapter. Institutions 
collected data related to students’ academic intentions (3.25) and progress (3.76) more extensively 
than they collected information regarding students’ cognitive (1.68) or affective (1.87) 
competencies, or regarding former students’ civic/social roles (1.80) and other postcollege 
competencies (2.29). 

There were statistically significant differences among accrediting regions for all extent 
measures except academic intentions. Overall, institutions in the Middle States, Southern and 
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North Central accrediting regions had the most extensive profiles of data collection. Middle States 
institutions collected the most extensive information on students’ academic progress (3.90), 
civic/social roles (1.97) and postcollege competencies (2.41), were second highest on affective 
competencies (1.87), and had the highest index score for comprehensiveness of data collection 
(37). Southern Association institutions were highest on affective assessment (1.94), second 
highest for academic progress (3.77), cognitive assessment (1.75), and postcollege assessment 
(2.35), and were tied for the highest index score for comprehensiveness of data collection (37). 
North Central institutions were highest in collecting cognitive competencies (1 .78) and had the 
highest index scores for comprehensiveness (37) and timing (20) of data collection. Extent scores 
for New England institutions consistently fell in the middle range. Institutions belonging to the 
Northwest region had comparatively lower scores, while those in the Western accrediting region 
ranked lowest on five of the seven measures for which there were significant differences across 
regions. 



9.6.2 Regional Accreditation Affiliation and Stud ent Assessment Data Collection Methods 

Institutions’ methods of collecting student assessment data were represented by scores on 
two factors (student-centered methods and external methods), three variables that did not load on 
these factors (transcript analysis, external examinations and surveys/interviews of withdrawing 
students) and an additive index of the number of assessment instruments used (comprehensive 
tests or examinations from institutional, state or commercial sources). Mean scores and standard 
deviations for each of these methods for all institutions by accrediting region are displayed in Table 
9.27. 

Compared to Table 9.8, the inclusion of private institutions raised the mean score for use of 
student-centered data collection methods by all institutions. The general pattern of data collection 
method use remained the same. Statistically significant differences across accrediting regions were 
found for all six data collection methods considered in this analysis, although two (transcript 
analysis and student- centered methods) were small in magnitude and significant at only the .05 
level. 
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^abI^^9^2^^Studen^^Assessinen^^Dat^^Coll£ctioi^^lethod^^b2^^\ccr£ditin^^Regioii 



Data Collection Methods 


All 

Institutions 

N=1373 


Accrediting Region 


Middle 

States 

N=186 


North 

Central 

N=522 


New 

England 

N=87 


Northwest 

N=79 


Southern 

N=417 


Western 

N=82 


F 


1. Number of instruments 


9 


9 


10 


8 


9 


10 


7 


13.40“ 




(3) 


(3) 


(4) 


(3) 


(4) 


(3) 


(3) 




2. Transcript analysis 


2.16 


2.23 


2.06 


2.30 


2.35 


2.24 


2.00 


2.41* 




(1.14) 


(1.19) 


(1.07) 


(130) 


(130) 


(1.16) 


(1.14) 




3. External examinations 


2.02 


1.96 


2.02 


1.95 


1.90 


2.14 


1.72 


13.11** 




(.49) 


(.38) 


(.42) 


(.66) 


(.41) 


(36) 


(.48) 




4. Surveys/interviews of 


2.40 


2.58 


2.35 


2.41 


2.27 


2.46 


2.03 


4.32** 


withdrawing students 


(1.00) 


(.99) 


(.98) 


(1.05) 


(.92) 


(1.01) 


(1.03) 




5. Student-centered methods 


1.37 


1.37 


1.39 


1.39 


1.37 


137 


1.26 


238* 




(.30) 


(32) 


(30) 


(30) 


(36) 


(30) 


(32) 




6. External methods 


2.04 


2.06 


2.07 


1.95 


2.00 


2.07 


1.75 


531** 




(.57) 


(.61) 


(37) 


(31) 


(34) 


(38) 


(.44) 





*p < .05, **p < .01 

Note: Standard deviations are in parentheses. Differences across group means for accrediting region were estimated 
using one-way ANOVA. 



The most notable profile was that of institutions in the Western accrediting region. 
Compared to the other accrediting regions, these institutions had the lowest scores for all data 
collection methods. Institutions in the Southern and North Central regions had the highest means 
for three measures. Southern region institutions made the most extensive use of external 
examinations (2. 14) and were tied with North Central institutions for the use of external methods 
(2.07) and on the highest index score for the number of assessment instruments used (10). North 

i • - • f. 

Central region institutions were tied with New England institutions for most extensive use of 
student-centered methods (1.39). Middle States, New England and Northwest region institutions 
each had the highest scores on one data collection method measure; their scores on the re maining 
measures spanned the upper to lower middle range. 



9.6.3 Regional Accreditation Af filiation and Assessment Studies and Reports 

The nature and extent of assessment studies and reports conducted by institutions were 
represented by scores on two factors (curricular experience studies and co-curricular experience 
studies), a single variable that did not load on these factors (conducts no studies) and two additive 
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indices of the number of studies conducted and number of reports written. Mean scores and 
standard deviations for each of these variables for all institutions by accrediting region are 
displayed in Table 9.28. This table shows the same pattern of conducting assessment studies and 
producing assessment reports for all institutions as was displayed in Table 9. 1 1 for public 
institutions. There were no statistically significant differences in the number of assessment studies 
and reports produced among institutions in different accrediting regions. 



Table 9.28 Student Assessment Studies and Reports by Accrediting Region 







Accrediting Region 




All 


Middles 


North 


New 










Studies and Reports 


Institutions 


States 


Central 


Fnglanri 


Northwest 


Southern 


Western 


F 




N=1363 


N=187 


N=519 


N=81 


N=79 


N=416 


N=81 




1. Number of studies 


2 


2 


2 


2 


2 


2 


2 


32 




(2) 


(2) 


(2) 


(2) 


a) 


a) 


(2) 




2. Curricular experience 


.14 


.13 


.13 


.14 


.15 


.14 


.14 


.41 


studies 


(.18) 


(.18) 


(.17) 


(.19) 


(.18) 


(.18) 


(.18) ‘ 


3. Co-curricular experience 


.20 


.21 


.20 


.22 


.20 


.21 


.19 


.28 


studies 


(.23) 


(.24) 


(.23) 


(.25) 


(.25) 


(-22) 


(-23) 


4. Conducts no studies 


.38 


.42 


.40 


.33 


.39 


.34 


.31 


1.58 




(.48) 


(-50) 


(.49) 


(.47) 


(.49) 


(.47) 


(.47) 




5. Number of reports 


2 


3 


3 


2 


2 


2 


3 


1.31 




(0 


(2) 


(1) 


(1) 


0) 


(1) 


(1) 





Note: Standard deviations are in parentheses. Differences across group means for accrediting region were es timated 
using one-way ANOVA. •* 



Summary. Accrediting region affiliation was associated with significant differences in the 
extent to which institutions collect data on students. Institutions in the Middle S tates , Southern and 
North Central regions have the most extensive profiles of data collection. New England and 
Northwest region institutions are in the middle range of extent scores while Western institutions 
reported the least extensive data collection efforts overall. There were also significant differences 
in institutions’ use of data collection methods across accrediting regions. Once again, Southern 
and North Central region institutions have the highest use scores overall. New England, Middle 
States and Northwest region institutions tend to be in the middle range. Institutions in the Western 
accrediting region have the lowest use scores for every data collection measure considered in this 
analysis. Regional accreditation affiliation was not related to differences in the number of 
assessment studies and reports undertaken by institutions. This finding is likely a function of the 
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low level of institutional activity reported in this dimension of student assessment approach. Also, 
the nature of studies and reports considered are those that serve internal purposes. Consequently, 
they are unlikely to be related to external influences. 

9.7 Regional Accreditation Affiliation and Organizational/Administrative Support for Student 

Assessment 

In the following sections, we report the relationships between regional accreditation 
affiliation and two domains of organizational and administrative support for student assessment 
The tables in this section show mean scores for specific dimensions within each of these 
ins titutional domains. ANOVAs were used to identify statistically significant differences in these 
dimensions by accrediting region. 



9.7.1 Regional Accreditation Affiliation and Institutional Support Strategy for Student Assessmen t 
We e xamin ed the relationship of regional accrediting affiliation to two dimensions of 
institutions’ internal support strategies: the extent to which the mission statement emphasized 
student assessment and institutions’ purposes for conducting student assessment (internal 
purposes, accreditation self-study, state requirements). Table 9.29 displays mean scores and 
standard deviations for these support strategy dimensions for all institutions by accrediting region. 



Table 9.29 Institutional Support Strategy for Student Assessment by Accrediting^ Region 







Accrediting Region 


Institutional Support Strategy 


All 


Middle 


North 


New 










for Student Assessment 


Institutions 


States 


Central 


England 


Northwest 


Southern 


Western 


F 




N=1377 


N=190 


N=527 


N=86 


N=79 


N=419 


N=82 




1. Mission emphasis 


1.53 


1.60 


1.45 


1.42 


1.25 


1.69 


1.46 


6.39** 


(.86) 


(.82) 


(.87) 


(.76) 


(.96) 


(.82) 


(.96) 


2. Internal purposes 


2.50 


2.49 


2.47 


2.46 


2.39 


2.59 


2.39 


4.85** 


(.49) 


(-52) 


(.49) 


(-50) 


(.54) 


(.46) 


(.55) 


3. Accreditation purposes * 


3.61 


3.46 


3.67 


3.50 


3.59 


3.65 


3.46 


4.50** 


(.65) 


(.71) 


(.60) 


(.72) 


(.71) 


(.61) 


(.82) 


4. State purposes 


2.89 


2.79 


2.75 


2.65 


2.88 


3.24 


2.43 


12.90** 


(1.18) 


(1.16) 


(1.17) 


(1.28) 


(1-20) 


(1.05) 


(130) 



**p < .01 

Note: Standard deviations are in parentheses. Differences across group means for accrediting region were estimated 
using one-way ANOVA. 
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As would be expected, the inclusion of private institutions resulted in a lower mean score 
for the importance given to meeting state requirements as an assessment purpose than was reported 
for public institutions (compare to Table 9.14). There were statistically significant differences 
across accrediting regions in all four support strategy dimensions. Institutions in the Southern 
accrediting association had the strongest profile of institutional support strategy of all accrediting 
regions. They gave the most emphasis to student assessment in their mission statements (1.69), 
had the highest importance ratings for internal purposes (2.59) and state requirements (3.24), and 
second highest rating for accreditation requirements as purposes for student assessment 
Institutions in the North Central region gave the highest importance rating overall to accreditation 
requirements as an assessment purpose (3.67). Beyond that institutional support strategy scores 
for institutions in the North Central and Middle States accrediting regions were in die upper middle 
range overall, while those of institutions in the New England region were in the middle range. 
Institutions in the Northwest region gave the least emphasis to student assessment in their mission 
statements (1.25) and were tied with Western region institutions for the lowest importance rating 
given to internal improvement as an assessment purpose (239). Western region institutions also 
had the lowest importance ratings for accreditation requirements (3.46) and state requirements 
(2.43) as purposes for engaging in student assessment 

9.7.2 Regional j^ccreditation Affiliation and Institutional Leadership/Govemance for Student 
Assessment 

We examined the relationship of regional accreditation affiliation to six dimensions of 
institutions’ leadership and governance patterns for student assessment the institution-wide 
administrative and governance activities used to promote student assessment; degree of 
administrative and faculty support for student assessment; and four dimensions of the structure and 
process of planning and policy setting for assessment Mean scores and standard deviations for 
each of these leadership and governance dimensions for all institutions by accrediting region are 
displayed in Table 9.30. 
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TabIe_9 ; 30_^ssessinent_Leadership^nd_Governancebv_Accrediting_Re£ioii 







Accrediting Region 


Assessment Leadership and 


All 


Middle 


North 


New 










Governance 


Institutions 


States 


Centra] 


England 


Northwest 


Southern 


Western 


F 




N=1381 


N=188 


N=526 


N=85 


N=80 


N=420 


N=82 




1. Administrative and 


2.35 


2.23 


2.44 


2.20 


2.33 


2.29 


2.31 


1.13 


governance activities 


(1*22) 


(1.25) 


(1.14) 


(134) 


(1.15) 


(130) 


(131) 




2. Administrator and faculty 


17.05 


17.09 


17.06 


16.55 


17.25 


17.19 


16.48 


1.57 


support 


(2.76) 


(2.63) 


(2.72) 


(335) 


(1.94) 


(2.70) 


(331) 




3. Formal centralized policy 


.50 


.29 


.59 


.24 


.40 


.59 


.32 


22.07** 




(.50) 


(.45) 


(.49) 


(.43) 


(.49) 


(.49) 


(.47) 




4. Institution-wide planning 


.70 


.64 


.85 


.48 


.71 


.60 


.63 


20.42** 


group 


(.46) 


(.48) 


(.36) 


(30) 


(.46) 


(.49) 


(.49) 




5. Breadth of assessment 


4 


4 


4 


3 


4 


4 


4 


7.17** 


planning group 


(2) 


(2) 


(1) 


(1) 


(1) 


(2) 


(1) 




6. Number approving 


3 


3 


2 


2 


3 


3 


3 


1.54 


changes 


0) 


(2) 


(1) 


(1) 


(1) 


0) 


(2) 





**p < .01 

Note: Standard deviations are in parentheses. Differences across group means for accrediting region were estimated 
using one-way ANOVA. 



The mean score for administrative and faculty support for student assessment was slightly 
lower for public and private institutions combined compared to that for public institutions (17.05 
versus 17.18). There were no meaningful differences in the mean scores for other 
leadership/govemance dimensions reported by all institutions and by public institutions only. 

There were statistically significant differences among accrediting regions in three of the six 
leadership and governance dimensions. Institutions in the North Central and Southern regions 
were most likely to have a formal centralized student assessment policy (.59). Institutions in the 
Northwest and Western regions were somewhat less likely to have this type of institutional 
assessment policy (.40 and .32 respectively), while those in the Middle States and New England 
regions were least likely to do so (.29 and 24 ). Institutional members of the North Central 
accrediting association were also most likely to have an institution-wide group for student 
assessment planning (.85); they were followed in descending order by institutions in the 
Northwest (.71), Middle States (.64), Western (.63), Southern (.60) and New England (.48) 
regions. Finally, institutions in the New England accrediting region had fewer members on their 
assessment planning groups (3) than institutions in the other accrediting regions. 
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Summary . Regional accreditation affiliation was significantly related to most of the 
measures of organizational and administrative support for student assessment. Membership in the 
Southern accrediting region was particularly associated with the strength of internal support for 
student assessment. The North Centra] and Middle States regions were associated with moderately 
high institutional support strategy scores, New England and Northwest were in the middle to lower 
range of scores, and the Western region had the lowest scores overall. Statistically significant 
differences by accrediting region were evident in institutions’ use of formal centralized assessment 
policies and likelihood of using an institution-wide group to do student assessment p lanning 

9.8 Regional Accreditation Affiliation and Assessment Management Policies and Practices 

In this section we examine the relationships among regional accreditation affiliation and the 
assessment management policies and practices used by institutions. The following thirteen 
dimensions of assessment management policies and practices were considered in these analyses: 
institutional evaluation of student assessment plan; resource allocation practices; access to 
assessment information; distribution of assessment reports; student involvement policies; providing 
incentives for student involvement; professional development policies; requiring faculty t rainin g in 
assessment; student affairs policies; faculty evaluation policies; considering assessment skills in 
hiring; encouraging faculty to assess student performance; and academic planning and review 
policies. ANOVAs were used to identify statistically significant differences in the use of these 
policies/practices among accrediting regions. Table 9.31 displays the mean scores and standard 
deviations related to the use of these assessment management policies and practices for all 
institutions and by accrediting region. 

The average scores for institutional use of assessment management policies and practices 
for public and private institutions combined did not differ appreciably from scores reported for 
public institutions only (see Table 9.20). Statistically significant differences by accrediting region 
were observed for the use of all assessment management policies and practices but one— the 
consideration of assessment skills when hiring faculty. 
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Table__9 : 31__Assessinent_Management_Policies and Practices by Accrediting Region 







Accrediting Region 


Assessment Management 


All 


Middle 


North 


New 










Policies and Practices 


Institutions 


States 


Central 


England 


Northwest 


Southern 


Western 


F 




N=1363 


N=189 


N=517 


N=83 


N=78 


N=414 


N=83 




1. Conducted evaluation of 


.50 


.35 


.54 


.27 


.40 


.58 


.45 


10.82** 


assessment approach 


(.50) 


(.48) 


(30) 


(.44) 


(49) 


(.49) 


(30) 




2. Resource allocation 


1.18 


1.16 


1.16 


1.06 


1.38 


1.20 


1.17 


2.61* 


practices 


(46) 


(.45) 


(.44) 


(.25) 


(37) 


(30) 


(38) 




3. Access to information 


3.46 


3.63 


3.25 


3.31 


3.36 


3.68 


3.50 


3.93** 




(1.65) 


(1.61) 


(1.72) 


(135) 


(1.64) 


(136) 


(1.62) 




4. Distribution of reports 


2.43 


2.31 


2.49 


2.07 


2.59 


2.50 


2.20 


2.46* 




(137) 


(141) 


(133) 


(1.48) 


(1.62) 


(130) 


(1.48) 




5. Student involvement 


2.66 


2.38 


2.75 


2.37 


2.49 


2.81 


2.44 


11.04** 


policies 


(.86) 


(.91) 


(.81) 


(.88) 


(.83) 


(.82) 


(.94) 




6. Student incentives 


1.87 


1.64 


1.92 


1.46 


1.94 


1.97 


1.79 


3.58** 




(1-23) 


(1.09) 


(121) 


(1.03) 


(1.19) 


(133) 


(122) 




7. Professional development 


1.89 


1.56 


1.82 


2.06 


1.81 


2.00 


1.79 


6.45** 


policies 


(.79) 


(39) 


(.81) 


(.90) 


(.77) 


(.77) 


(.77) 




8. Faculty training required 


2.47 


2.09 


2.41 


2.48 


2.39 


2.79 


1.88 


10.85** 




(136) 


(1.43) 


(134) 


(1.47) 


(130) 


(1.62) 


(128) 




9. Student affairs policies 


1.94 


1.77 


1.85 


1.73 


2.19 


2.13 


1.88 


4.54** 




(1.18) 


(1.11) 


(1.10) 


(1.10) 


(121) 


(130) 


(1.13) 




10. Faculty evaluation policies 


1.24 


1.36 


1.16 


1.19 


1.26 


1.31 


1.17 


3.88** 




(.66) 


(.69) 


(39) 


(.62) 


(.61) 


(.71) 


(.70) 




11. Hiring process 


1.68 


1.66 


1.66 


1.75 


1.96 


1.71 


1.39 


2.21 




(1.10) 


(1.05) 


(1.10) 


(1.18) 


(121) 


(1.14) 


(.73) 




12. Encourage faculty to 


3.99 


3.82 


4.11 


3.95 


4.18 


4.03 


3.25 


6.77** 


assess 


(131) 


(138) 


(123) 


(130) 


(1.07) 


(133) 


(131) 




13. Academic planning and 


2.79 


2.40 


3.10 


2.64 


2.32 


2.74 


2.68 


16.28** 


review policies * 


(.97) 


(1.04) -i 


(91) •• 


(.94) 


(1.03) 


(•91) 


(.99) 





* p < .05, **p < .01 

Note: Standard deviations are in parentheses. Differences across group means for accrediting region were es timated 
using one-way ANOVA. 



Overall, institutions in the Southern accrediting region reported the most extensive use of 
assessment management policies and practices. They were most likely to have evaluated their 
student assessment plan (.58) and scored highest on the use of policies/practices related to access 
to assessment information (3.68), student involvement (2.81), offering incentives to students 
(1.97), and requiring faculty to have assessment training (2.79). They had the second highest use 
scores for policies and practices related to resource allocation (1.20); distribution of assessment 
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reports (2.50); professional development for faculty (2.00) and student affairs personnel (2.13); 
faculty evaluation (1.31); and academic planning and review (2.74). 

There was considerable variability in the patterns of using these policies and practices in the 
other five accreditation regions. Scores for institutions belonging to the Middle States, North 
Central, New England and Northwest accrediting regions ranged from the highest to the lowest for 
at least one policy/practice each, with the majority of use scores distributed over the midrange. 

Only institutions in the Western region did not have any use scores that were highest or second 
highest overall; their scores ranged from the middle to the lowest across accrediting regions. 

9.9 Regional Accreditation Affiliation and Institutional Uses and Impacts of Student Assessment 

Finally, we examined the relationships among regional accreditation affiliations and the 
extent to which institutions had used and documented impacts of student assessment information 
Factor analysis distinguished two dimensions of institutional decision making that could be 
influenced by student assessment information: academic decisions concerning instructional 
experiences, academic plans or structures, student assessment planning, and resource allocation; 
and faculty decisions. Three dimensions of institutional impact were identified: faculty impacts 
(e.g., satisfaction, interest in teaching, teaching methods); student impacts (e.g., retention or 
graduation, grade performance, satisfaction); and external impacts (e.g., external reputation or 
image, external funding received). ANOVAs were used to identify statistically significant 
differences in the assessment uses and impacts by accrediting region. Table 9.32 displays the 
mean scores and standard deviations related to uses and impacts for all institutions by accrediting 
region. 

The average scores for institutional uses and impacts of student assessment information for 
public and private institutions combined did not differ appreciably from scores reported for public 
institutions only (see Table 9.23). There were statistically si gnifi cant diff erences in all five 
measures of institutional uses and impacts, but differences associated with impacts on faculty were 
small. Once again, institutions belonging to the Southern accrediting region had the strongest 
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Table^ i 32_^^nstitutional^ses^ndIm£acts > j)£^tudeii^^ssessnientb^^Accreditin^^tegioii 







Accrediting Region 




All 


Middle 


North 


New 










Uses and Impacts 


Institutions 


States 


Central 


England 


Northwest 


Southern 


Western 


F 




N=1310 


N=177 


N=504 


N=79 


N=76 


N=398 


N=79 




1. Academic decisions 


1.40 


1.36 


1.36 


1.32 


1.43 


1.51 


1.31 


8.07** 




(.41) 


(.40) 


(.40) 


(.38) 


(.40) 


(.40) 


(.41) 




2. Faculty decisions 


1.28 


1.29 


1.18 


1.25 


1.24 


1.40 


1.29 


6.08** 




(.62) 


(.59) 


(.54) 


(.60) 


(33) 


(.69) 


(.72) 




3. Faculty impacts 


1.57 


1.48 


1.61 


1.36 


1.61 


1.63 


1.48 


2.75* 




(.77) 


(.76) 


(.75) 


(.73) 


(.79) 


(.77) 


(.82) 




4. Student impacts 


1.62 


1.55 


1.55 


1.36 


1.65 


1.80 


1.62 


7.06** 




(.80) 


(.77) 


(.78) 


(.68) 


(.77) 


(.84) 


(.78) 




5. External impacts 


1.17 


1.10 


1.13 


1.07 


1.17 


1.27 


1.13 


4.46** 




(.54) 


(.50) 


(31) 


(35) 


(33) 


(38) 


(34) 





*p < .05, **p < .01 

Note: Standard deviations are in parentheses. Differences across group means for accrediting region were estimated 
using one-way ANOVA. 



profile. They reported the greatest influence of assessment data on academic decisions (1.5 1) and 
faculty decisions (1.40) and were most likely to have documented positive impacts from 
assessment information on student performance (1.80) and external performance indicators (1.27). 
Institutions in the Northwest region had the second highest profile of scores for assessment uses 
and impacts, followed by institutions in the Middle States, North Central and Western regions. 
Institutions in the New England accrediting region were least likely to report using and 
documenting impacts from assessment information; they had the lowest scores of all regions on 
positive impacts of assessment on student performance (1.36) or external performance (1.07) 
indicators. 



9. 10 Summary of External Influences on Institutional Student Assessment Patterns 
Distinctive relationships were observed between state and regional accreditation influences 
and institutions* patterns of student assessment Overall, it appears that having a centralized, 
authoritative governance structure for higher education (consolidated governing board or 
coordinating regulatory board) is positively associated with the degree of institutional support for 
student assessment Institutions in states with these forms of governance structures for higher 
education have more extensive student assessment approaches, have established more internal 
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strategies to support assessment, and report greater internal support for assessment. This greater 
degree of involvement is more likely if the initiatives used to frame assessment requirements are 
non-legislative or a combination of statute and policy, and if reporting requirements exist, but 
institutions are permitted to develop and report their own student performance indicators and 
outcomes. State assessment approaches were only minimall y related to institutions’ reported use 
and impacts of student assessment 

Compared to state assessment approaches, regional accreditation affiliation was associated 
with more and larger differences in how institutions approach, support and use student 
assessment. This finding is consistent with institutions’ perceptions of the relative influence of 
state and accreditation requirements on their assessment activities, and is congruent with prior 
research. Compared to other accrediting regions, institutions in the Southern, North Central and 
Middle States regions reported higher scores for their student assessment approaches and 
organizational /administrative support for student assessment. This finding is likely due to the 
longer period of time during which these associations have been active in promoting student 
assessment as an important institutional activity. Institutions from the Southern accrediting region 
had the strongest profile of using assessment management policies and practices to support student 
assessment, and reported the most extensive uses and impacts of student assessment information. 
The Southern accrediting region was among the first to incorporate criteria related to student 
assessment in its self-study requirements. Further, a number of southern states have been active in 
student assessment initiatives. 
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10. Relationship of Institutional Student Assessment Patterns to Institutional 
Uses and Impacts of Student Assessment 

In this chapter, we examine the relationship of institutions’ student assessment approaches 

and patterns of organizational and administrative support for student assessment to institutional 

uses and impacts of student assessment information (research question seven). In the tables that 

follow, we present correlations among these domains of institutional student assessment activity 

for all institutions and by institutional type. For these analyses, we employed indices of the 

variables as described in chapter eight. Due to the large sample size, virtually all variables were 

significantly correlated. To focus our discussion, we will consider only those correlations greater 

than 30. All correlations were significant at the p < .01 significance level unless otherwise 
indicated. 

10. 1 Relationship of Student Assessment Approach to Assessment Uses and Impacts 
Institutional approach to student assessment was represented by the following derived 
variables: the extent to which institutions collected data on former students’ competencies and 
current students cognitive and affective competencies; comprehensiveness of data coll ect ion 
efforts, number of assessment instruments used; extent of use of student-centered and external 
assessment methods; number of studies conducted concerning the relationship of students’ 
curricular and co-curricular experiences to student performance; and total number of assessment 
studies conducted and assessment reports produced. 

10 - 11 Relationship Of Student Assessment Appr oach to Assessment Uses 

Two factors reflected different dimensions of student assessment information use in 
institutional decision making. “Academic decisions” refers to the use of assessment information in 
institutional decisions concerning academic planning, resource allocation and instructional 
experiences. “Faculty decisions” refers to the use of assessment information in institutional 
decisions concerning faculty promotion or rewards. Table 10.1 displays the correlations greater 
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than .30 among these assessment approach variables and institutional uses of assessment 
information. 

For all institutions, six of the eleven measures of institutions’ student assessment 
approaches had correlations greater than .30 with the use of student assessment information in 
academic decisions: collection of cognitive data, comprehensiveness of data collection overall, 
number of assessment instruments used, use of student~centered assessment methods, number of 
studies of students’ curricular experiences conducted and total number of assessment studies 
conducted. Conversely, collecting data on students’ postcollege competencies, current students’ 
affective competencies, and from external constituents; conducting studies of students’ co- 
curricular experiences; and the total number of assessment reports produced were not strongly 
correlated with the use of assessment information in academic decisions. In part, these results are 
attributable to the low incidence of institutional activity in these assessment approach Himp ncinnc 
Further, it may be expected that data regarding former students or external constituencies may be 
less mfluential in institutional decision making than data collected from current students. There 
were no correlations greater than .30 between approach measures and the use of student 
assessment information in faculty promotion decisions. 

An examination of correlations for each of the five types of institutions shows four 
assessment approach indices consistently had strong correlations with academ ic decision uses of 
assessment information: extent of assessing students’ cognitive competencies, overall 
comprehensiveness of data collection, number of studies of students’ curricular experiences and 
number of assessment reports produced. Differences in correlations between assessment approach 
measures and assessment information uses were also evident across institutional types. 

The profile of correlations for associate of arts institutions differed from that of most other 
institutional types in two ways: the collection of data from former students was 
strongly correlated with using assessment information for academic decisions, while the use of 
external evaluation methods was not There were no strong correlations between approach 
dimensions and faculty decision uses of assessment information. 
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Institutional Uses of Student Assessment Information 
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The profile of correlations for baccalaureate institutions differed from those in most other 
institutions in three respects. The number of studies of students’ co-curricular experiences and 
performance was not strongly correlated to using assessment information for academic decisions 
but the number of assessment reports produced was. Further, the use of student-centered and 
external assessment methods, and conducting studies of students’ curricular experiences were 
correlated with the use of assessment information in faculty promotion and reward d ec isions. 
These correlations between assessment approaches and faculty decisions are congruent with the 
emphasis of baccalaureate institutions on excellence in undergraduate education. 

Master’s institutions differed little from other institutions in the correlations existing 
between assessment approach measures and academic decision uses of assessment information. 
The number of assessment instruments and use of external assessment methods were not strongly 
correlated with using assessment information in academic decisions. T ilrp h^a l ^ireate 
institutions, the number of assessment reports produced was correlated with use of assessment 
information in academic decisions. There were no strong correlations between assessment 
approach dimensions and faculty decision uses of assessment information. 

Doctoral institutions had the fewest correlations between assessment approaches and use of 
assessment information in academic decisions. Unlike most other institutions, the total number of 
assessment instruments used and the use of student-centered assessment methods were not related 
to this use of assessment data. Somewhat surprisingly, doctoral institutions were the only 
institutional type besides baccalaureate institutions to have correlations between a ssessme nt 
approach measures and the use of assessment information in faculty decisions. The cor related 
measures were the same for these two institutional types: use of student-centered and prim al 
assessment methods, and number of studies conducted of the relationship between students’ 
curricular experiences and performance. 

Research institutions had the most correlations between assessment approach measures and 
use of assessment data for academic decisions. As was the case with associate of arts institutions, 
the collection of data from former students was correlated with academic decision-making uses. 
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There were no strong correlations between assessment approach measures and the use of 
assessment information in faculty decisions. 



10 - 1 - 2 Relationship of Student Assessment Approach to Assessment Impart* 

Table 10.2 displays correlations greater than .30 among assessment approach variables and 
three institutional impacts of assessment information: faculty impacts (e.g., interest in teaching, 
change in teaching methods), student impacts (e.g., retention or graduation rates, grade 
performance) and external impacts (e.g., funding received from external sources, external 
reputation). 



Examining data for all institutions reveals that two assessment approach indices were 
strongly correlated with faculty impacts from student assessment: the number of studies analyzing 
the relationship between students’ curricular experiences and performance and the total number of 
assessment studies conducted. No dimensions of assessment approaches were strongly correlated 
with student or external impacts. 

The pattern of correlations between assessment approach indices and assessment impacts 
differed considerably across the five types of institutions. Associate of arts institutions had the 
fewest strong correlations between the assessment approach and impact variables. Studies of 
students’ co-curricular experiences and performance, and the total number of assessment studies 
conducted were related to faculty impacts from assessment There were no strong correlations 
between assessment approach indices and student or external impacts from assessment 
Baccalaureate institutions had a larger number of strong associations between their assessment 
approaches and impacts. Indices concerning the number of studies conducted of students’ 
institutional experiences and performance were related to documented faculty and student 
from assessment Conducting studies of students’ curricular experiences and total number of 
assessment studies conducted were strongly associated with faculty and student impacts; the 
number of co-curricular experience studies conducted was also strongly related to achieving 
positive faculty impacts. The extent of collecting data on students’ cognitive and affective 
competencies was related to external impacts from assessment 
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Institutional Impacts of Student Assessment Information* 
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Master s institutions had neither the most nor the fewest relationships between assessment 
approaches and impacts. Conducting studies of students’ curricular experiences and the number of 
studies conducted were strongly correlated with faculty and student impacts from assessment. The 
number of co-curncular experience studies conducted was also strongly correlated with student 
impacts. There were no strong correlations between assessment approach indices and external 
impacts from assessment. 

Doctoral institutions had the largest number of strong correlations between assessment 
approach indices and impact measures. Studies of students’ curricular experiences and total 
number of studies were related to faculty impacts as were the extensiveness of collecting data on 
students’ affective domains and overall comprehensiveness of data collection efforts. Conducting 
studies of aspects of students’ curricular and co-curricular experiences and student performance 
were related to documenting positive student and external impacts from assessment In addition, 
using external assessment methods and the number of assessment reports produced were 

associated with external impacts. 

$ 

; Research institutions had a unique pattern of relations between assessment approaches and 
faculty impacts. Three indices reflecting the content and extent of data collection were strongly 
related to faculty impacts: extent of collecting post college data and cognitive data and overall 

comprehensiveness of data collection efforts. Total number of assessment instruments was also 

1(1 { 1 • , ; 

strongly related. Unlike other institutional types, conducting studies of students’ institutional 
experiences and performance were not strongly related to achieving faculty impacts. There were 
no strong associations between assessment approach indices and student impacts from assessment. 
Conducting studies of students’ co-curricular experiences and the total number of studies 
conducted were related to external impacts from assessment 



10.1.3 Summary of Assessment Approaches with Uses and Impacts 

Relationships are apparent between several dimensions of institutions’ student asses sment 
approaches and their uses and impacts of student assessment information. Two related approach 
indices — studies of the relationship of students’ curricular experiences and performance, and the 
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total number of assessment studies conducted — had strong correlations with most of the 
assessment use and impact factors. Otherwise, patterns of correlations with assessment approach 
variables varied for specific assessment uses and impacts. 

For each of the five types of institutions, several approach indices were consistently 
strongly correlated with institutions’ use of student assessment information in academic decisions. 
These approach measures reflect the extent to which cognitive data on current students is collected, 
the comprehensiveness of data collection efforts, the use of student-centered assessment methods, 
and institutional efforts to analyze the relationship between students’ institutional experiences and 
performance. The lower occurrence of strong correlations between assessment approaches and the 
use of assessment information in faculty decisions reflects the resolve of most institutions to keep 
student assessment participation and information separate from faculty evaluation and reward 
processes. Baccalaureate and doctoral institutions differed from other institutional types in this 
respect In these institutions, student-centered assessment methods that require faculty 
participation and studies examining the relationship between students' curricular experiences and 
performance were related to the use of assessment information in faculty decisions. 

Strong correlations between assessment approaches and the impact variables were most 
frequent for faculty impacts and least frequent for external impacts. Conducting studies that 
analyze relationships between students’ curricular and co-curricular experiences and their 
performance was quite consistently related to documenting assessment-related impacts on faculty 
performance, and was also related, to a lesser extent, to student and external impacts. This 
reinforces the assertion of assessment scholars that institutions must not only collect student 
assessment data but must also support efforts to analyze and interpret the meaning of this data if 
assessment is to contribute to improvements in institutional performance. To a lesser extent, 
indices regarding the content and extent of data collection efforts were strongly related to achieving 
faculty and external impacts from assessment 
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10.2 Relationship of Organizational and Administrative Support to Assessment Uses and Impacts 

Organizational and administrative support for student assessment was represented by the 
following variables and indices: mission emphasis on undergraduate education and assessment; 
importance of conducting assessment for internal purposes; importance of conducting assessment 
for accreditation purposes; importance of conducting assessment to meet state reporting 
requirements; the number of administrative and governance activities undertaken to promote 
assessment; and the degree of administrative and faculty support for assessment. 

10.2.1 Relationship of Organizational and Administrative Support to Assessment Uses 

Correlations greater than .30 between dimensions of organizational and administrative 
support for assessment and the use of assessment information in academic and faculty decisions 
are displayed in Table 10.3. Examining the results for all institutions shows that four of the six 
support variables/indices were strongly correlated with using assessment information in academic 
decisions: conducting assessment for internal purposes, conducting assessment for state 
requirements, administrative and governance activities promoting assessment, and administrative 
and faculty support for assessment There were no strong associations between dimensions of 
organizational and administrative support for assessment and the use of assessment information in 
faculty decisions. 

One organizational and administrative support variable was correlated with using student 
assessment information in academic decisions was observed in each of the five types of institutions 
— conducting assessment for internal purposes. Another support measure — administrative and 
faculty support for assessment — was correlated with academic decision uses of assessment 
information in all but doctoral institutions. There were no strong correlations between 
organizational and ad minis trative support measures and faculty decision uses of assessment 
information common to all five institutional types. It is also interesting to note that two support 
variables — mission emphasis and conducting assessment for accreditation purposes — were not 
strongly correlated with uses of student assessment information for either academic or faculty 
decisions in any of the five institutional types. 
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Associate of arts and research institutions had the same profile of correlations between 
support and use variables. In both types of institutions, conducting assessment for internal 
purposes, and administrative and faculty support for assessment were strongly correlated with 
using assessment information in academic decisions. There were no strong correlations between 
support measures and faculty decision uses of assessment information. 

Baccalaureate and master’s institutions also shared the same pattern of correlations between 
organizational and administrative support indices and assessment use measures. In both types of 
institutions, conducting assessment for internal purposes, the number of administrative and 
governance activities promoting assessment, and the extent of administrative and faculty support 
for assessment were strongly associated with academic decision uses of assessment information. 
There were no strong correlations between organizational and administrative support measures and 
the use of assessment information. 

1 Doctoral institutions differed from other types of institutions in their profile of correlations. 
Only one support measure was strongly related to academic decision uses of assessment 
information: conducting assessment for internal purposes. This institutional type was the only one 

in* ^4 
+* *■% 

in which any organizational and administrative support measure — the number of administrative 
and governance activities promoting assessment — was strongly correlated with faculty decision 
uses of assessment information. 

10.2.2 Relationship of Organizational and Administrative Support to Assessmentimpacts 

Table 10.4 displays correlations greater than .30 among organizational and administrative 
support measures and the three impacts of assessment information for all institutions and by 
institutional type. 

As Table 10.4 shows, there were very few strong correlations between these two domains. 
Considering all institutions together, only one support measure — the number of administrative 
an d governance activities promoting assessment — was strongly correlated with positive faculty 
impacts from assessment There were no strong relationships between organizational and 
administrative support measures and either student or external impacts. 
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Among associate of arts institutions, the number of administrative and governance activities 
promoting assessment was strongly associated with achieving positive faculty and external impacts 
from assessment. In baccalaureate institutions, administrative and governance activities promoting 
assessment were strongly correlated only with faculty impacts. In master’s and research 
institutions, there were ho strong relationships among organizational and administrative support 
measures and assessment impacts. Doctoral institutions had the largest number of strong 
associations between the domains of organizational and administrative support and assessment 
impacts. Three support measures, were strongly related to faculty impacts: mission emphasis, 
conducting assessment for internal purposes, and the number of administrative and governance 
activities promoting assessment. The number of administrative and governance activities 
promoting assessment was also strongly related to positive student impacts from as sessme nt 



10.2.3 Summary of Organizational and Administrative Support with Uses and Impacts 

There were a limited number of strong associations among measures of organizational and 
administrative support for assessment and institutional uses and impacts of assessment 
information. One support measure — the number of administrative and governance activities 
promoting student assessment — was associated with several of the use and impact factors. 
Compared to the assessment approach variables, there appear to be weaker relationships between 
variables in this conceptual domain and the likelihood of institutions using and being positively 
impacted by assessment information. 

Strong correlations consistently emerged between the use of assessment information in 
academic decisions and two measures of organizational and administrative support conducting 
assessment for internal purposes and, in all but doctoral institutions, the degree of administrative 
and faculty support for assessment Other analyses, not displayed here, showed these two support 
measures were correlated with each other (r 2 = .36; p < .01). These findings are congruent with 
scholars’ contentions that conducting assessment for internal rather than external purposes may be 
more likely to promote internal support for assessment and encourage institutional use of 
assessment information (Braskamp, 1991; Ewell, 1987a; Hutchings & Marchese, 1990; Sell, 
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1989a). The number of administrative and governance activities undertaken to promote assessment 
was strongly related to academic decision uses of assessment information in baccalaureate and 
master’s institutions, and to faculty decision uses in doctoral institutions. With the exception of 
this latter finding, there were no strong relationships between organizational and administrative 
support measures and the use of assessment information in faculty decisions. 

Strong associations were observed between achieving positive faculty impacts from student 
assessment and one support measure — the number of administrative and governance activities 
promoting assessment — in associate of arts, baccalaureate and doctoral institutions. This finding 
suggests the important role such institutional initiatives may play in encouraging faculty to be 
involved in and make use of student assessment. Otherwise, there was a dearth of strong 
relationships between organizational and administrative support measures and either student or 
external impacts from assessment. 

10.3 Relationship of Assessment Management Policies and Practices to Assessment Uses and 

Impacts 

Assessment management policies and practices were represented by eleven indices and 
variables: resource allocation practices; the use of assessment information to decide budget 
allocations for academic units; breadth of internal access to assessment information on individual 
students; distribution of assessment reports; policies promoting student involvement in assessment 
activities; provision of incentives for student involvement in assessment; policies providing 
professional development on assessment for academic administrators and faculty; requiring faculty 
to have training in student assessment; policies providing professional development on assessment 
for student affairs administrators and staff; inclusion of assessment-related criteria in faculty 
evaluation policies; and incorporating assessment information in academic planning and review 
processes. 



10.3.1 Relationship of Assessment Management Policies and Practices to Assessment Uses 

Table 10.5 displays correlations greater than .30 between assessment management policies 
and practices and the use of assessment information in academic and faculty decisions for all 
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institutions and by institutional type. Seven assessment management policies and practices were 
strongly correlated with the use of assessment information in academic decisions in all institutions: 
the distribution of assessment reports; policies promoting student involvement in assessment; 
policies providing professional development on assessment for academic administrators and 
faculty; providing professional development on assessment for student affairs personnel; requiring 
faculty to have assessment training; including assessment-related criteria in faculty evaluation 
policies; and incorporating assessment information in academic planning and review processes. 

One assessment management policy dimension — the inclusion of assessment-related criteria in 
faculty evaluation policies — was strongly correlated with using assessment information in faculty 
decisions in all but research institutions. 

Considering the profile of correlations within each institutional type, three assessment 
management policies and practices were consistently strongly related to academic decision uses of 
assessment information: providing professional development on assessment for faculty and 
academic administrators, providing professional development on assessment for student affairs 
personnel, and incorporating assessment information in academic planning and review processes. 
Beyond these three similarities, the patterns of correlations between assessment management 
policies and practices and assessment uses differed for each of the five types of institutions. 

Associate of arts institutions had comparatively few strong correlations between the 
assessment management and assessment use domains. In addition to the three correlations 
discussed above, academic decision uses of assessment information in associate of arts institutions 
was strongly related to the distribution of assessment reports and the inclusion of assessment- 
related criteria in faculty evaluation policies. Faculty evaluation criteria was the only assessment 
management measure strongly correlated with faculty decision uses of assessment information. 

Baccalaureate institutions had eleven strong correlations between the domains of 
assessment management policies and practices and assessment uses. They were the only 
institutional type in which a strong correlation existed between resource allocation practices and 
academic decision uses. In addition to the three correlations common to all institutional types. 
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Table 10.5 Correlations of Assessment Management Policies with Practices and Institutional Use of Assessment Information by 
Institutional Type 
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other assessment management measures strongly correlated with academic decision uses were: 
access to assessment information, policies promoting student involvement in assessment, including 
assessment-related criteria in faculty evaluation policies, and using student assessment information 
in academic planning and review processes. Four assessment management measures were 
strongly associated with faculty decision uses of assessment information in baccalaureate 
institutions: access to assessment information, professional development on assessment for 
academic administrators and faculty, assessment-related criteria included in faculty evaluation 
policies, and incorporating assessment information in academic planning and review processes. 

In master’s institutions, in addition to the correlations between assessment management 
policy measures and academic decisions common to all five types of institutions, two other strong 
correlations emerged: policies promoting student involvement and requiring faculty training in 
assessment. Only one assessment management policy measure was strongly associated with the 
use of assessment information in faculty decisions: the inclusion of assessment-related criteria in 
faculty evaluation policies. 

Of all types of institutions, doctoral institutions had the greatest number of strong 
correlations between the domains of assessment management policies and assessment uses. In 
addition to the correlations between assessment management measures and academic decisions 
common to all institutional types, four other assessment management measures were strongly 
associated with this assessment use: access to assessment information, policies promoting student 
involvement, requiring faculty training in assessment, and including assessment-related criteria in 
faculty evaluation policies. Five assessment management variables/indices were strongly 
correlated with faculty decision uses: providing incentives for student involvement in assessment, 
requiring faculty training in assessment, providing professional development on assessment for 
student affairs personnel, including assessment-related criteria in faculty evaluation policies and 
incorporating assessment information in academic planning and review processes. These findings 
suggest that assessment management policies and practices play a particularly important role in 
promoting the use and impact of student assessment information in doctoral institutions. 
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Research institutions had comparatively few strong correlations between assessment 
management policies and practices and assessment uses. There were strong correlations between 
academic decision uses of assessment information and distribution of reports, student involvement 
policies, professional development on assessment for academic administrators and faculty, 
professional development on assessment for student affairs personnel, requiring faculty training, 
and incorporating assessment information in academic planning and review processes. No 
assessment management policies or practices were strongly associated with faculty decision uses. 
Unlike other institutional types, the inclusion of assessment-related criteria in faculty evaluation 
policies was not strongly related to either academic or faculty decision uses of assessment 
information. This result may be reflective of the comparatively strong research emphasis and high 
degree of faculty autonomy in these institutions. 

10.3.2 Relationship of Assessment Management Policies and Practices to Assessment Impacts 

Table 10.6 displays correlations greater than .30 between assessment management policies 
and practices and faculty, student and external impacts of assessment information for all 
institutions and by institutional type. 

Incorporating assessment information into academic planning and review processes was 
strongly correlated with faculty, student and external assessment impacts for all institutional 
respondents. This was the only strong association between the variables of assessment 
management policies and practices and assessment impacts. The academic planning and review 
index was associated with achieving faculty and student impacts in all five types of institutions 
except research universities. Otherwise, patterns of correlations varied considerably by 
institutional type. 

Among associate of arts institutions, incorporating assessment information in academic planning 
and review processes was associated with each of the three impact measures. In addition, 
providing professional development on assessment for academic administrators and faculty was 
related to faculty impacts; providing professional development on assessment for student affairs 
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Institutional Impacts of Student Assessment Information* 
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personnel was related to student impacts; and resource allocation practices was associated with 
achieving external impacts from assessment. 

In baccalaureate institutions, three assessment management measures were correlated with 
faculty impacts: providing professional development on assessment for academic administrators 
and faculty, requiring faculty to have training in assessment and using assessment information in 
academic planning and review processes. As was the case for associate of arts institutions, 
providing professional development on assessment for student affairs personnel and using 
assessment information in academic planning and review processes were associated with achieving 
student impacts from assessment. Providing professional development on assessment for student 
affairs personnel and including assessment-related criteria in faculty evaluation policies were 
strongly correlated with positive external impacts from assessment. 

There were few correlations greater than .30 between assessment management policies and 
practices and assessment impact measures in master’s institutions. Incorporating assessment 
information in academic planning and review processes was associated with positive faculty, 
student and external impacts from assessment. In addition, requiring faculty training in assessment 
was correlated with faculty impacts and the distribution of assessment reports was associated with 
external impacts. 

Among doctoral institutions, seven different assessment management policies and practices 
had correlations greater than .30 with one or more assessment impact measures. Access to 
assessment information, providing incentives for student involvement in assessment, professional 
development on assessment for student affairs personnel, including assessment-related criteria in 
faculty evaluation policies, and incorporating assessment information in academic planning and 
review processes were strongly associated with all three assessment impacts. In addition, 
providing professional development on assessment for academic administrators and faculty was 
correlated with achieving faculty impacts while having policies promoting student involvement in 
assessment was associated with student impacts. 
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In contrast, only one assessment policy measure was strongly associated with each of the 
three impact measures in research institutions. Breadth of internal access to individual student 
assessment information was strongly correlated with faculty impacts. Including assessment-related 
criteria in faculty evaluation policies was strongly correlated with student impacts. Requiring 
faculty training in assessment was strongly correlated with external impacts from assessment. 

10.3.3 Summary of Assessment Management Policies with Uses and Impacts 

Strong relationships are evident between several dimensions of institutions’ assessment 
management policies and their uses and impacts of student assessment information. One 
assessment management index — the incorporation of assessment management information in 
academic planning and review processes — was strongly associated with all five use and impact 
measures. Overall, assessment management policies were more often strongly associated with 
assessment uses than with assessment impacts; and with academic decision uses than with faculty 
decision uses. The profile of correlations between assessment management policies and 
assessment uses and impacts varied considerably for each of the five types of institutions. 

A large proportion of assessment management policies and practices had correlations 
greater than .30 with using assessment information in institutional decisions. The strongest 
correlation overall was between incorporating assessment information into academic planning and 
review processes and the use of assessment ‘information in academic decisions (correlations ranged 
from .54 to .66). Further, providing professional development on assessment for academic 
administrators and faculty and for student affairs personnel were consistently correlated with 
academic decision uses. This finding suggests that providing professional development on 
assessment orientates institutional personnel to the potential uses of assessment information. With 
the exception of research institutions, including assessment-related criteria in faculty evaluation 
policies was also strongly correlated with academic decision uses. Other correlations between 
assessment management measures and academic decisions varied by type of institution. 

There were fewer correlations between assessment management measures and faculty 
decision uses of assessment information. The inclusion of assessment-related criteria in faculty 
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evaluation policies was associated with using assessment information in faculty decisions in all 
types of institutions except research institutions. This linkage between evaluation criteria and 
evaluation decisions makes intuitive sense. Baccalaureate and doctoral institutions had the greatest 
number of correlations between assessment management measures and assessment uses. This 
finding suggests these institution types have made greater use of assessment management policies 
and practices to support the use of assessment information. 

Compared to correlations with assessment use measures, there were fewer correlations 
greater than .30 among assessment management policies and practices and assessment impacts. 
With the exception of research institutions, incorporating assessment information into academic 
planning and review processes was the assessment management policy most consistently 
associated with achieving positive impacts from assessment. In all but master’s and research 
institutions, providing professional development on assessment for academic administrators and 
faculty was correlated with achieving positive faculty impacts from assessment and providing 
professional development on assessment for student affairs personnel was correlated with positive 
student impacts from assessment Doctoral institutions had the greatest number of strong 
correlations between assessment management measures and all three assessment impacts. This 
. result suggests that for these institutions, assessment management policies and practices may be a 
particularly important means of realizing positive impacts from assessment Conversely, there 
appear to be weak relationships between assessment management practices and assessment impacts 
in research institutions. 



10.4 Summary of Internal Relationships to Assessment Uses and Impacts 
Several variables/indices from each of the three internal influence domains — institutional 
approach to assessment, organizational and administrative support for assessment, and assessment 
management policies and practices — emerged as having strong associations with assessment uses 
and impacts. In the institutional approach to assessment domain, the extensiveness of institutions’ 
data collection efforts, and specifically, the extent to which institutions collected data on students’ 
cognitive domains were strongly associated with both using and realizing impacts from assessment 
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information. In the organizational and administrative support domain, conducting assessment for 
internal purposes and the degree of administrative and faculty support for assessment were 
strongly associated with using assessment data for academic decisions. Slightly less frequently, 
administrative and governance activities promoting assessment were associated with academic 
decision uses and faculty impacts. In the assessment management policies and practices domain, 
professional development on assessment for academic administrators and faculty, professional 
development on assessment for student affairs personnel, and incorporating assessment 
information in academic planning and review processes were strongly associated with academic 
uses. Including assessment-related criteria in faculty evaluation policies was correlated with 
faculty decision uses in all but research institutions. Incorporating assessment information into 
academic planning and review processes was correlated with assessment impacts in all types of 
institutions except research institutions. 

Overall, variables from the three internal influence domains were more often strongly 
associated with assessment use measures than with assessment impact measures. This finding is 
partly indicative of the generally low frequency of institutions documenting impacts attributable to 
their assessment efforts. Variables/indices from the domains of institutional approaches to 
assessment and assessment management policies and practices were more often correlated with 
assessment uses and impacts than were variables/indices from the domain of organizational and 
administrative support for assessment. 

Differences were evident in patterns of correlations among internal influence and 
assessment use and impact variables within specific types of institutions. Baccalaureate and 
doctoral institutions had the largest number of correlations greater than .30 among these domains, 
particularly for assessment uses and impacts related to faculty. Conversely, research institutions 
had the fewest strong correlations among internal influence domains and assessment uses and 
impacts. 
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11. External and Institutional Influences on Institutional Uses and Impacts of Student 

Assessment 

In this chapter, we address research question eight. Using the indices and derived variables 
identified in chapter eight, we conducted multivariate analyses to examine the relationship of external 
influences, institutional characteristics, institutional approaches to assessment, and organizational and 
administrative support for student assessment variables to five indices measuring the institutional uses 
and impacts of student assessment information. Multivariate analyses were conducted in two stages. In 
the first stage, regression models were estimated for all institutional respondents. In the second stage, 
separate regression models were estimated for each institutional type. In the sections that follow, we (1) 
review the methodology of our multivariate analyses, (2) report results from regression models estimated 
for all institutions, (3) report results from regression models estimated separately by institutional type and 
(4) compare patterns of significant predictors for each of the five assessment use and impact measures 
across institutional types. 



11.1 Method of Regression Analyses 

A series of stepwise regression analyses were conducted to test the multivariate relationships 
specified in the conceptual model for this study. In this analytical method, independent variables enter 
into the regression model according to the amount of unique variance each explains in the outcome 
measure. The order of entry of independent variables is determined statistically rather than conceptually. 
Only statistically significant predictors are retained in the model. 

The use of stepwise regression was justified on several counts. First, the conceptual and 
empirical literature suggested a large number of external and internal influences on assessment uses and 
impacts but provided no basis for ordering the entry of predictor variables into the model a priori. 
Second, this study collected cross-sectional rather than longitudinal data; therefore, it was not possible to 
infer causal relationships among the predictor variables. Finally, two other regression methods were 
tried: entering all variables in the model and entering variables in three discrete blocks (institutional 
characteristics, external influences and institutional approach to assessment). Results obtained were not 
substantially different from those obtained using the stepwise method. Stepwise regression produces a 
reduced model of statistically significant predictors and calculates the change in explained variance in the 



outcome measure associated with each retained variable. Thus, it seemed most useful in identifying the 
most important predictors from among a large number of potential influences. 

Stepwise regression analyses were conducted for five dependent variable indices measuring 
institutional uses and impacts of student assessment: academic decisions, faculty decisions, faculty 
impacts, student impacts and external impacts. Independent variables and indices identified in chapter 
eight were drawn from each of the following conceptual domains: external influences, institutional 
characteristics, institutional approach to student assessment, organizational and administrative support for 
assessment, and assessment management policies and practices. In preparation for regression analyses, 
two categorical independent variables — accrediting region and institutional type — were treated as 
dichotomous variables and one category from each was omitted from the analyses. The Northwest 
accrediting region was the omitted category for accrediting regions. Master’s institutions was the omitted 
category for institutional type. These categories fell closest to the mean in scores on the dependent 
variables in regression analyses. We merged doctoral with research institutions to increase sample sizes. 
Post-hoc analyses of ANOVA results did not show significant differences between these two institutional 
types on the dependent variables. Because of the large number of predictors in our model, mean 
substitution was used to replace missing data in the independent variables. Mean substitution values 
corresponded to the sample being analyzed in the model. In models estimated for all institutions, mean 
replacement values were calculated from data on all institutional respondents. In models estimated for 
separate types of institutions, mean replacement values were calculated for each institutional type. With 
the exception of three variables (budget decisions— missing 36%; state initiative— missing 21%; 
administrative and governance activities— missing 21%), the amount of missing data for any variable in 
the analyses did not exceed 14%. A complete description of variables used in regression analyses is 
presented in Table 11.1. 

Table 11.1 Variables used in Regression Analyses 



Variable 



Type of 
Variable 



Values 



Data Source 



Institutional Characteristics 
enrollment 
institutional type 



item 



item 



Associate of Arts 
Baccalaureate 
Master's 
Doctoral 



IPEDS 



IPEDS 



ERiC 




314 
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— Research 

Variable Type of Values Data Source 

Variable 

External Influences on Student Assessment 



accrediting region 


item 

(dummied) 


Middle States 
North Central 
New England 
Northwest 
Southern 
Western 


IPEDS 


accrediting purpose 


item 


Scale range 2 = 1-4 


ISSA 3 


accrediting influence 


item 


1 = negative influence 

2 = not a factor 

3 = either a reason to initiate 

or to increase 
involvement 

4 = both a reason to initiate 


ISSA 



and to increase 
involvement 



state initiative 


item 


1 = No state plan 

2 = State policy 

3 = State statute 

4 = Combination of policy 

& statute 


SAS 4 


state approach 


item 


1 = No indicators or 

outcomes 

2 = Institutional specific 

3 = Common for some 

4 = Common for all 


SAS 


state purpose 


item 


Scale range 2 = 1-4 


ISSA 


Institutional Approach to Student Assessment 


postcollege assessment 


factor 


Alpha = .83 
Scale range 5 = 1-4 
Mean = 2.27 


ISSA 


cognitive assessment 


factor 


Alpha = .71 
Scale range 5 =1-4 
Mean = 1.62 


ISSA 


affective assessment 


factor 


Alpha = .68 
Scale range 5 = 1-4 
Mean = 1.74 


ISSA 


number of instruments 


additive index 


Range = 0-24 
Mean = 9.35 


ISSA 


student-centered methods 


factor 


Alpha = .61 
Scale range = 1-4 6 
Mean = 1.37 


ISSA 


■: best copy available 


ERJC 


265 


! : 315 





Variable 



Type of 
Variable 



Values 



Data Source 



external methods 


factor 


Alpha = .63 
Scale range = 1-4 6 
Mean = 2.04 


ISSA 


number of studies 


additive index 


Range = 0-9 
Mean = 2.20 


ISSA 


number of reports 


additive index 


Range = 0-5 
Mean = 2.47 


ISSA 


Institutional Support for Student Assessment 








mission emphasis 


additive index 


Range = 0-3 
Mean = 1.48 


ISSA 


internal purposes 


factor 


Alpha = .79 
Scale range 3 =1-4 
Mean = 2.48 


ISSA 


administrative and governance activities 


additive index 


Range = 0-7 
Mean = 2.33 


ISSA 


administrative and faculty support 


additive index 


Range = 4-20 
Mean = 17.05 


ISSA 


formal centralized policy 


item 


1 = yes/ 0 = no 


ISSA 


institution wide planning group 


item 


1 = yes / 0 = no 


ISSA 


AcademicJManagement Policies and Practices. 








budget decisions 


additive index 


Range = 0-2 
Mean = .08 


ISSA 


computer support 


additive index 


Range = 0-3 
Mean = .79 


ISSA 


access to information 


additive index 


Range = 0-5 
Mean = 3.46 


ISSA 


distribution of reports 


additive index 


Range = 0-6 
Mean = 2.43 


ISSA 


student involvement 


factor 


Alpha = .69 
Scale Range = 1-5 7 
Mean = 2.66 


ISSA 


professional development 


factor 


Alpha = .77 
Scale Range = 1-5 7 
Mean =1.89 


ISSA 


student affairs 


factor 


Alpha =.84 
Scale Range = 1-5 7 
Mean = 1.94 


ISSA 
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Variable 


Type of 
Variable 


Values 


Data Source 


faculty evaluation 


factor 


Alpha = .77 
Scale Range = 1-5 7 
Mean = 1.24 


ISSA 


academic planning and review policies 

Institutional Uses and Impacts of Student 
Assessment 


factor 


Alpha = .84 
Scale Range = 1-5 7 
Mean = 2.79 


ISSA 


academic decisions 


factor 


Alpha = .83 
Scale Range = 1-4 8 
Mean = 1.40 


ISSA 


faculty decisions 


factor 


Alpha = .79 
Scale Range = 1-4 8 
Mean = 1.28 


ISSA 


faculty impacts 


factor 


Alpha = .79 
Scale Range = 1-4 9 
Mean = 1.57 


ISSA 


student impacts 


factor 


Alpha == .82 
Scale Range = 1-4 9 
Mean = 1.62 


ISSA 


external impacts 


factor 


Alpha = .82 
Scale Range = 1-4 9 
Mean = 1.17 


ISSA 



^tegrated Postsecondary Education Data System 

2 1 = no importance, 2 = minor importance, 3 = moderate importance, 4 = very important 

3 Inventory of Institutional Support for Student Assessment 

“Assessment of Teaching and Leamingjfor Improvement and Public Accountability: State Governing, Coordinating Board 
and Regional Accreditation Association Policies and Practices (Cole, Nettles, & Sharp, 1997) 

5 1= not collected, 2 = collected for some, 3 = collected for many, 4 = collected for all students 

6 1 = not used, 2 = used in some units, 3 = used in most units, 4 = used in all units 

7 1 = not done at all, 2 = done in a few departments, 3 = done in some departments, 4 = done in many departments, 5 = done 
in most departments 

*1 = no action or influence unknown, 2 = action taken, data not influential, 3 = action taken, data somewhat influential, 4 = 
action taken, data very influential 

*1 = not monitored, do not know, 2 = monitored, negative impact, 3 = monitored, no known impact, 4 = monitored, 
positive impact 



A total of twenty-five separate regression analyses were conducted for this study. In the 
first stage, five analyses — one for each of the five dependent use and impact measures — were 
conducted using data from all institutional respondents. In the second stage, twenty regression 
analyses were conducted. Within each of four institutional types (associate of arts, baccalaureate, 
master’s, and research and doctoral), analyses were run for each of the five dependent use and- 
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impact measures. With the exception of excluding institutional type, the predictors used in these 
analyses were identical to those used for all respondents. Results obtained from these regression 
analyses are discussed below. 

11.2 External and Institutional Influences on Institutional Uses and Impacts of Student 

Assessment for All Institutions 

Results of stepwise regression analyses for each of the five use and outcome measures 
(academic decisions, faculty decisions, faculty impacts, student impacts and external impacts) for 
all institutions are summarized below. Complete results of these analyses appear in Table 1 1.2. 

The R 2 is provided for each model. This statistic is the multiple coefficient of determination and 
represents the total amount of variance accounted for in the outcome measure by the model. The 
beta coefficient (B), its significance (p < .05 or .01), direction of influence (positive or negative) 
and the associated proportion of change in variance in the outcome measure (AR 2 ) is provided for 
each independent variable in the model. 

11.2.1 Influences on Academic Decisions for All Institutions 

The factor representing institutional use of student assessment information for academic 
decisions was regressed against forty independent variables (see Table 1 1 .2). This outcome 
measure reflects the extent to which student assessment information influenced the following 
academic decisions: creating or modifying instructional experiences; developing or revising 
academic plans or structures; designing or reorganizing student affairs units; and allocating 
resources to academic units. For this analysis, the factor index for academic planning and review 
policies was omitted as an independent variable since many of the items comprising this index were 
quite similar to the items comprising the dependent measure for academic decisions. As the R 2 
value shows, this model predicted a substantial proportion of the variance in institutional use of 
assessment information for academic decisions (R 2 = .41). 

Fifteen external and institutional variables or indices emerged as statistically significant 
predictors of this use measure. Of these, conducting assessment for the purpose of internal 
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improvement was the strongest predictor (8 = .14, p < .01, AR 2 = .14) followed by having 
policies involving student affairs administrators and staff in professional development for student 
assessment (8 = .12, p < .01, AR 2 = .09) and the total number of assessment studies conducted 
(8 = .14, p < .01, AR 2 = .06). Other variables that contributed 2% or greater change in total 
variance were policies involving students in assessment activities (8 = .10, p < .01, AR 2 = .03), 
policies linking student assessment to faculty evaluation (8 = .1 1, p < .01, AR 2 = .03) and the 
distribution of student assessment reports (8 = .10, p < .01, AR 2 = .02). The remaining 
statistically significant predictors (belonging to the North Central accrediting region, belonging to 
the Southern accrediting region, collecting data on cognitive competencies, using student-centered 
assessment methods, number of administrative and governance activities promoting assessment, 
extent of administrator and faculty support for assessment, having evaluated the assessment 
approach, computer support for assessment information, and professional development on 
assessment for administrators and faculty) did not contribute more than 1% of the variance in 
academic decisions. 

The largest number of significant predictors (six) came from the domain of assessment 
management policies and practices. Together, these variables accounted for 19% of the variance in 
academic decision uses although most was attributable to professional development policies for 
student affairs personnel. This finding underlines the importance of assessment management 
policies and practices as mechanisms for integrating assessment activities into institutional 
functioning and decision making. The domain of organizational and administrative support for 
assessment accounted for 17% of the variance and contributed four significant predictors. This 
finding suggests the importance of institutional leadership promoting assessment as an 
institutionally-relevant and valued activity if assessment results are to be used in academic decision 
making. Institutional approach to student assessment accounted for 8% of the variance with the 
collection of data on students’ cognitive competencies, use of student-centered assessment 
methods, and particularly, conducting studies of student assessment data emerging as important 
influences. In comparison, the domains of institutional characteristics and external influences 



appear to be less influential in determining the extent of institutional use of student assessment 
information in academic decisions. 

11.2.2 Influences on Faculty Decisions for All Institutions 

Faculty decisions is a factor index reflecting the extent to which student assessment 
information influenced decisions regarding faculty promotion, tenure, salary increases or rewards 
(see Table 1 1.2). This outcome measure was regressed against forty independent variables. For 
this analysis, the factor for faculty evaluation policies regarding student assessment was excluded 
from the model because several items comprising this factor were quite similar to those in the 
outcome measure. Compared to the first analyses discussed, this model explained relatively little 
variance (R 2 = .15). This result can be attributed to the relatively low use of assessment 
information in faculty-related decisions among all institutions. 

Ten variables/indices emerged as statistically significant predictors of institutions using 
student assessment information to make faculty decisions. Incorporating assessment information 
into academic planning and review processes was the strongest predictor (6 = .1 1, p < .01, AR 2 = 
.05) followed by using student-centered assessment methods (6 = .1 1, p < .01, AR 2 = .03) and 
the total number of assessment studies conducted (6 = .10, p < .01, AR 2 = .02). The seven 
remaining statistically significant variables (being in the North Central accrediting region, 
baccalaureate institutional type, using external assessment methods, having an institution-wide 
assessment planning group, using assessment information to make budget decisions, access to 
assessment information, and professional development policies on student assessment for faculty 
and administrators) each accounted for no more than 1% of change in the variance of faculty 
decisions. 

In this model, the largest number of significant predictors (four) were from the domain of 
assessment management policies and practices and these accounted for 8% of the variance in 
faculty decision uses. This finding suggests, as above, the important connection between 
embedding student assessment within assessment management policies and practices — 
particularly using assessment results to inform academic planning and review. The domain of 



institutional approach to student assessment provided three significant predictors and accounted for 
6% of the variance. In particular, the use of student-centered assessment methods which often 
require significant faculty involvement in their development and interpretation, and the conduct of 
studies examining the relationship between students’ institutional experiences and academic 
performance, were associated with the likelihood of institutions linking assessment information 
with decisions concerning faculty. The remaining domains of organizational and administrative 
support, institutional characteristics and external influences were not important influences on this 
outcome measure, each contributing only one significant predictor and accounting for 1 % or less of 
the variance. 

11.2.3 Influences on Faculty Impacts for All Institutions 

Faculty impacts is a factor index reflecting the extent to which student assessment 
information had a positive impact on faculty satisfaction, discussions of undergraduate education, 
interest in teaching, and changes in teaching methods. This index and the remaining two measures 
of assessment impacts were regressed against forty-one independent variables (see Table 1 1 .2). 
These predictors explained 26% of the variance in faculty impacts of student assessment (R 2 = 
.26). 

Ten variables/indices were statistically significant predictors of achieving positive faculty 
impacts. The strongest of these was the total number of assessment studies conducted (B = .18, p 
< .01, AR 2 = .10) and incorporating student assessment information into processes for academic 
planning and review (B = .1 1, p < .01, AR 2 = .06). To a lesser extent, the number of 
administrative and governance activities used to promote student assessment (B = .13,p < .01, 

AR 2 = .03) and policies providing professional development on assessment for academic 
administrators and faculty members (B = .12,p < .01, AR 2 = .02) contributed to faculty impacts. 
None of the remaining statistically significant predictors (baccalaureate institutional type, collecting 
data on students’ cognitive competencies, having an institution-wide assessment planning group, 
having evaluated the assessment approach, distribution of assessment reports, and having policies 
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linking faculty evaluation to student assessment) contributed more than 1% to the change in 
variance in faculty impacts. 



The domains of assessment management policies and practices, and organizational and 
administrative support contributed the largest number of significant predictors (four and three 
predictors, respectively). They accounted for 10% and 5% of the variance in faculty impacts thus 
providing some support for the importance of these two domains in promoting the impact of 
student assessment information. However, a single institutional approach to student assessment 
index, the number of assessment studies conducted, emerged as the strongest predictor overall 
accounting for 10% of the variance. The domains of external influences and internal characteristics 
had comparatively little influence on achieving faculty-related impacts from assessment. 

11.2.4 Influences on Student Impacts for All Institutions 

Student impacts is a factor index reflecting the extent to which student assessment 
information had a positive impact on students’ retention or graduation, grade performance, 
achievement on external examinations and satisfaction (see Table 1 1.2). This model accounted for 
21% of the variance in this outcome measure (R 2 = .21). 

Eight variables/indices were statistically significant predictors of student impacts. Of these, 
the strongest was incorporating assessment information into academic planning and review 
processes (B = .17, p < .01, AR 2 =.11) followed by the total number of assessment studies 
conducted (B = .16, p < .01, AR 2 = .04) and policies concerning professional development on 
assessment for student affairs administrators or staff (B = .10, p < .01, AR 2 = .02). The remaining 
significant predictors (belonging to the Southern accrediting region, having evaluated the 
assessment approach, computer support for assessment information, distribution of assessment 
reports and policies linking assessment to faculty evaluation decisions) each contributed no more 
than 1% to the change in the variance of student impacts. 

Assessment policies and practices produced the greatest number (five) of significant 
predictors and accounted for 17% of the variance. In particular, linking assessment information to 
academic planning and review processes appeared as the strongest determinant of institutions 
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achieving positive assessment-related impacts on students’ performance. In a related vein, 
involving student affairs professional in learning about assessment was conducive to using 
assessment information to improve student performance. The institutional approach domain 
contributed only one significant, but important, predictor. Once again, institutions that undertake 
analyses of the relationships between aspects of students’ institutional experiences and their 
academic performance were more likely to document positive student impacts from assessment 
than those that conducted no such studies. The domains of organizational and administrative 
support for assessment and external influences each contributed one significant but comparatively 
less important predictor, while no institutional characteristics were significant predictors of student 
impacts. 



11.2.5 Influences on External Impacts for All Institutions 

External impacts is a factor index reflecting the extent to which student assessment 
information had a positive impact on external indicators of institutional performance such as state 
funding allocation, accreditation evaluations and institutional reputation (see Table 1 1.2). This 
model accounted for 19% of the variance in this outcome measure (R 2 = .19). 

Eleven variables were statistically significant predictors. The strongest of these was 
incorporating assessment information into academic planning and review processes (6 = . 1 1 , p < 
.01, AR 2 = .09), total number of assessment studies conducted (6 = .1 1, p < .01, AR 2 = .04), and 
to a lesser extent, linking assessment to faculty evaluation criteria (6 = .12, p < .01, AR 2 = .02) 
and the number of assessment reports distributed (6 = .08, p < .01 , AR 2 = .02). The remaining 
significant predictors (conducting assessment for state purposes, total number of assessment 
instruments used, mission emphasis, having evaluated the assessment approach, using assessment 
information to make budget decisions, computer support for assessment information, and policies 
involving students in assessment) each contributed 1% or less to the change in variance in external 
impacts from assessment. 

The domain of assessment management policies and practices provided the greatest number 
of significant predictors of external impacts (six) which accounted for 16% of the variance. This 
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finding reinforces the importance of embedding assessment activities within ongoing assessment 
policies and processes if institutions are to reap observable assessment-related impacts. The 
institutional approach variable of number of assessment studies conducted was again an important 
predictor. The domains of organizational and administrative support made little significant 
contribution to realizing positive external impacts. Two variables, mission emphasis and 
evaluating the institution’s student assessment process, contributed only 2% of the variance. 
Conducting assessment for state purposes was the only external influence variable that was a 
significant predictor; it contributed less than 1% of change in the variance. No institutional 
characteristics were significant predictors of external impacts. 



11.2.6 Summary of Regression Analyses for All Institutions 

Based on regression results for data from all institutions, it appears this model is most 
useful for predicting institutional use of student assessment information to make academic 
decisions. To a lesser extent, it is capable of predicting the extent to which institutions will achieve 
faculty impacts, student impacts, and external impacts from assessment, or will use assessment 
information to make faculty-related decisions. These differences in predictive capacity are partly 
due to the distribution of these outcome measures across all institutions. 

The profile of statistically significant predictors varied somewhat for each outcome measure 
considered. However, several variables emerged as important predictors across all outcome 
measures. Most prominent across the analyses of all five use and impact variables were 
incorporating assessment information into academic planning and review processes and the total 
number of assessment studies conducted. The consistent appearance of these variables as strong, 
significant predictors in each analysis in which they were included suggests these are essential 
means by which institutions can promote the use of assessment information to improve institutional 
performance. Other variables that emerged as important predictors were having evaluated the 
assessment approach, distribution of assessment reports, and linking assessment to faculty 
evaluation policies. Conversely, variables related to external influences and institutional 
characteristics explained little of the variance in these outcome measures. 
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11.3 External and Institutional Influences on Institutional Uses and Impacts of Student 

Assessment by Institutional Type 

In the second stage of multivariate analyses, regression analyses were conducted for each 
of the five dependent use and impact measures within the following types of institutions: associate 
of arts, baccalaureate, master’s, and research and doctoral. Based on an ANOVA post-hoc 
analysis, a comparison of scores on the dependent variables for research and doctoral institutions 
revealed no significant differences; thus, they were combined for these analyses to increase the 
number of cases available. The following tables present the statistically significant predictors for 
the five dependent use and impact measures by institutional type. 



11.3.1 External and Institutional Influences on Institutional Uses and Impacts of Student 
Assessment for Associate of Arts Institutions 

Table 1 1.3 presents the statistically significant predictors for the five regression analyses 
conducted for associate of arts institutions. 

Academic Decisions . The first analysis, in which academic decisions was regressed 
against the independent variables, predicted 41% of the variance in this outcome measure (R 2 = 
.41). This amount was equivalent to the associated model estimated for all institutional types. 
Eleven variables/indices were statistically significant predictors (at the p < .01 or p < .05 
significance level) of institutions using assessment information to make academic decisions. 
Having policies concerning professional development on assessment for student affairs 
administrators or staff was the strongest of these (6 = .16, p < .01, AR 2 = .14). Even though it 
had a lower level of statistical significance than some of the other predictors, collecting data on 
students’ cognitive competencies accounted for the second largest change in variance of academic 
decision uses (6 = . 10, p < .05, AR 2 = .09). Other strong predictors of using assessment 
information for academic decisions were the total number of assessment studies conducted (6 = 
.16, p < .01, AR 2 = .05), conducting assessment for internal purposes (6 = .12, p < .01, AR 2 = 
.03), the number of assessment reports distributed (6 = .14, p < .01, AR 2 = .02), using student- 
centered assessment methods (6 = .13, p < .01, AR 2 = .02) and belonging to the Southern 
accrediting region (B = . 1 2, p < .0 1 , AR 2 = .02). Collecting data on former students’ 
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Table 11.3 External and Internal Influences on Institutional Uses and Impacts of Student 
Assessment for Associate of Arts Institutions (N=548) 

Institutional Uses and Impacts of Student Assessment 



Academic Faculty Faculty Student External 

Decisions Decisions Impacts Impacts Impacts 



R 2 


.41 


** 


.12** 


.28 


** 


.22 


** 


.23 


** 




Beta 


££? 


Beta 


AR 2 


Beta 




Beta 


AR 2 


Beta 


AR 2 


External Influences 
North Central 
Southern 
State approach 


.12** 


.02 


-11** 


.02 


-.09* 


.01 










Accreditation purposes 
Institutional Approach 
Cognitive assessment 


.10* 


.09 


-.09* 


.01 


-.12** 


.01 










Post-college assessment 
Number of instruments 


.08* 


.01 














.09* 


.01 


Student-centered methods 


.13** 


.02 






.09* 


.01 










Number of studies 


.16** 


.05 


.16** 


.03 


.14** 


.06 


.14** 


.03 


.10* 


.01 


Organizational A 

Administrative Support 
Internal purposes 
Admin. & gov. activities 
Institution-wide planning 
group 


.12** 


.03 


-.09* 


.01 


.17** 


.04 






.12** 


.03 


Conducted evaluation 
Assessment Management 
Policies & Practices 










.08* 


.01 










Academic planning & 


n/inc 




.15** 


.05 


.10* 


.02 


.21** 


.13 


.13** 


.10 


review 






















Budget decisions 


















.08* 


.01 


Computer support 


.10** 


.01 






.09* 


10 1 


.15** 


;04 


.18** 


.05 


Distribution of reports 


.14** 


.02 


















Student involvement 


.08* 


.01 














.10* 


.01 


Professional development 
Student affairs 


.16** 


.14 


.12** 


.01 


.17** 


.11 


.13** 


.02 






Faculty evaluation 


.10** 


.01 


n/inc 




.11* 


.01 


.11** 


.01 


.13** 


.02 



*p < .05; **p < .01 



competencies, computer support for assessment information, having policies to involve students in 
assessment activities, and linking assessment with faculty evaluation policies were also statistically 
significant predictors but contributed 1% or less to the variance in academic decision uses. 
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As was the case in the overall model for academic decisions, the assessment management 
policies and practices domain contributed many of the significant predictors of this outcome 
measure. Almost as important were variables in the institutional approach to assessment domain. 
Conversely, the domain of organizational and administrative support for assessment contributed 
only one significant predictor. This finding suggests that for associate of arts institutions, the 
extensiveness and nature of data collection and analysis efforts is more likely to promote 
institutional use of assessment information for academic decision making than are patterns of 
administrative and governance support activities. 

FacultvJDecisions . In the second analysis, the model predicted only 12% of the variance in 
using assessment information to make faculty decisions (R 2 = .12). This result is partly 
attributable to the relative infrequency of this use of assessment information among associate of 
arts institutions (see Table 7.2). Six variables emerged as statistically significant predictors of this 
assessment use. Incorporating assessment information into processes for academic planning and 
review was the strongest positive predictor (6 = . 15, p < .01, AR 2 = .05) followed by the total 
number of assessment studies conducted (6 = .16, p < .01, AR 2 = .03) and having professional 
development policies on assessment for academic administrators and faculty (6 = .12, p < .01, AR 2 
= .01).There were three negative predictors of this use of assessment information: belonging to 
the North Central accrediting region (6 = -. 1 1, p < .01, AR 2 = .02), conducting assessment for 
accreditation purposes (6 = -.09, p < .05, AR 2 = .01) and having an institution-wide assessment 
planning group (6 = -.09, p < .01, AR 2 = .01). However, given the small amount of variance 
explained by these variables, their influence is clearly minimal. 

Faculty Impacts . The regression analysis of faculty impacts explained 28% of the variance 
in this outcome measure (R 2 = .28). Ten variables/indices emerged as statistically significant 
predictors of positive impacts from assessment on faculty. For associate of arts institutions, 
providing professional development on assessment for academic administrators and faculty was the 
strongest positive predictor, contributing more than a third of the explained variance in faculty 
impacts (B = .17,p < .01, AR 2 = .11). This finding suggests the provision of professional 
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development is a powerful way for these institutions to promote the use of student assessment 
among faculty. Other strong positive predictors included the total number of assessment studies 
conducted (B = . 14, p < .01, AR 2 = .06), number of administrative and governance activities 
undertaken to promote student assessment (B = .17, p < .01, AR 2 = .04), and incorporation of 
assessment information into academic planning and review processes (B = .10, p < .01, AR 2 = 
.02). Six other predictors, although statistically significant, each contributed only 1% of the 
variance in faculty impacts. Four were positive predictors: linking assessment to faculty 
evaluation policies, computer support for assessment information, using student-centered 
assessment methods and evaluating the assessment approach. Two were negative predictors: 
conducting assessment for state purposes and for accreditation purposes. These negative 
relationships, although of small magnitude, suggest that conducting assessment for external 
purposes may be incompatible with promoting faculty support and use of assessment in associate 
of arts institutions. 

Student Impacts . The regression model predicting positive impacts from assessment on 
student performance explained 22% of the variance in this outcome measure (R 2 = .22). Five 
variables/indices were statistically significant predictors. The most important of these was the 
extent to which institutions incorporated assessment information into academic planning and 
review processes (B = .21, p < .01, AR 2 = .13); this index contributed more than half of the 
explained variance in the model. Other predictors were, in descending order of their contribution 
to the R 2 statistic, computer support for assessment information (B = .15, p < .01, AR 2 = .04), 
total number of assessment studies conducted (B = .14, p < .01, AR 2 = .03), professional 
development policies on assessment for student affairs administrators and staff (B = .13, p < .01, 
AR 2 = .02) and policies linking assessment to faculty evaluation (B = .11, p < .01, AR 2 = .01). 

External Impacts . The results of this final regression analysis show the model was slightly 
better at predicting external impacts from assessment in associate of arts institutions than was the 
case for all institutions (R 2 = .23). Eight variables/indices were statistically significant predictors. 
Chief among these was incorporating assessment information into academic planning and review 
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processes (6 = .13, p < .01, AR 2 = .10) followed in importance by computer support for 
assessment information (8 = .18,p < .01, AR 2 = .05), number of administrative and governance 
activities used to promote assessment (8 = . 12, p < .01, AR 2 = .03), and linking assessment to 
faculty evaluation policies (8 = .13, p < .01, AR 2 = .02). The remaining significant predictors 
(number of assessment instruments used, number of assessment studies conducted, use of 
assessment information for budget decisions, and policies involving students in assessment) 
contributed 1% each to the variance in external impacts. 

Summary . In general, the models predicting the five outcome measures worked as well for 
associate of arts institutions as they did for respondents from all types of institutions. Based on the 
R 2 values for each analysis, the model for academic decisions worked best at predicting its 
outcome measure. Several variables/indices appeared as significant predictors across all or several 
outcome measures. As was observed in the previous analyses of data from all institutions, 
incorporating assessment information into academic planning and review processes was a 
consistently strong predictor of assessment uses and impacts followed by the total number of 
assessment studies conducted. For associate of arts institutions, computer support for assessment 
information also emerged as an important predictor as did linking assessment with faculty 
evaluation policies. Assessment management policies and practices and, to a lesser degree, 
organizational and administrative support were the conceptual domains that contributed the greatest 
number of significant predictors. Variables related to the institutional approach to assessment also 
figured quite extensively as predictors in several of the models. 

11.3.2 Regression Results for Baccalaureate Institutions 

Table 1 1.4 presents the statistically significant predictors for each of the five regression 
analyses conducted for baccalaureate institutions. 

Academic Decisions . The model predicting the use of assessment information for academic 
decisions in baccalaureate institutions explained 40% of the variance in this outcome measure (R 2 = 
.40). Seven variables/indices were statistically significant predictors of this assessment 
information use. Professional development policies on assessment for academic administrators 
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Table 11,4 External and Internal Influences on Institutional Uses and Impacts of Student 
Assessment for Baccalaureate Institutions (N=316) 









Institutional Uses and Impacts of Student Assessment 








Academic 


Faculty 


Faculty 


Student 


External 




Decisions 


Decisions 


Impacts 


Impacts 


Impacts 


R 2 


.40 


»* 


.31** 


.29 


* * 


.24** 


.22 


» * 




Beta 


Mi 


Beta AR 2 


Beta 


Mi 


Beta AR 2 


Beta 


AR 2 


External Influences 
North Central 






-.14** .02 












State purposes 






.21** .02 












Accrediting influence 

Institutional 

Characteristics 






-.17** .04 








-.12* 


.02 


Control (l=pub, 2=priv) 
Institutional Approach 






.12* .01 












Cognitive assessment 
Number of instruments 


.13** 


.02 


.17** .03 








.17** 


.02 


Student-centered methods 






.17** .02 












External methods 
Number of studies 
Organizational & 
Administrative Support 


.16** 


.03 


.17** .06 


.24** 


.14 


.19** .05 






Mission emphasis 
Internal purposes 


.19** 


.09 




.10* 


.01 




.11* 


.01 


Admin. & gov. activities 
Administrator & faculty 
support 

Formal centralized policy 
Institution-wide planning 






.13** .02 


.21** 


.04 


-.12* .01 


-.12* 


.01 


group 

Conducted evaluation 






-.10* .01 


.11* 


.01 


.13* .01 


.16** 


.03 


Assessment Management 
Policies & Practices 
Academic planning & 


n/inc 






.22** 


.07 


.24** .12 


.16** 


.10 


review 

Budget decisions 
Access to information 
Student involvement 


.16** 


.03 


.14** .09 


-.11* 


.01 








Professional development 


.12* 


.17 














Student affairs 


.12* 


.01 








.18** .03 






Faculty evaluation 


.18** 


.05 


n/inc 








.14* 


.03 



*p < .05; **p < .01 
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and faculty was only significant at the .05 level but accounted for the largest proportion of the 
variance in academic decisions (6 = .12, p < .05, AR 2 = .17). Conducting assessment for the 
purpose of internal improvement (6 = .19, p < .01, AR 2 = .09) and policies linking assessment and 
faculty evaluation (6 = .18, p < .01, AR 2 = .05) also contributed positively to the explained 
variance, followed by the total number of assessment studies conducted (6 = .16,/? < .01, AR 2 = 
.03), policies involving students in assessment activities (6 = .16, p < .01, AR 2 = .03), the number 
of assessment instruments used (6 = .13, p < .01, AR 2 = .02) and finally, policies providing 
professional development on assessment for student affairs personnel (6 = .12, p < .05, AR 2 = 
. 01 ). 

These findings suggest that baccalaureate institutions that provide support through 
assessment management policies and practices, particularly those concerning professional 
development, will make greater use of assessment information in academic decisions. For these 
institutions, it may be more important to emphasize professional development policies directed at 
academic administrators and faculty and less important to emphasize policies related to student 
affairs administrators and staff. It also appears that baccalaureate institutions who view internal 
improvement as an important purpose of assessment will be more likely to use collected 
assessment information in institutional decision making. 

Faculty Decisions . The regression analysis examining the use of assessment information 
for faculty decisions was better at predicting this outcome measure in baccalaureate institutions than 
was the case for other types of institutions. This difference is partly due to the finding that 
baccalaureate institutions were more likely than other types of institutions to report using 
assessment information to make faculty decisions (see Table 7.2). Ten statistically significant 
predictors were identified; of these, seven were positive predictors and three were negative 
predictors. Together, these variables/indices explained 31% of the variance in faculty decisions 
(R 2 = .31). 

The positive predictor accounting for the most variance in faculty decision uses of 
assessment information was the breadth of access among internal personnel to assessment 
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information on individual students (8 = .14,/? < .01, AR 2 = .09). This was the only significant 
predictor from among assessment management policies and practices variables. Other positive 
predictors were, in descending order of explained variance in faculty decisions, the use of external 
assessment methods (8 = .17, p < .01, AR 2 = .06), collecting data on students’ cognitive 
competencies (8 = .17, p < .01, AR 2 = .03), conducting assessment for state purposes (8 = .21, p 
< .01, AR 2 = .02), using student-centered assessment methods (8 = .17, p < .01, AR 2 = .02), 
extent of administrator and faculty support for assessment (8 = .13, p < .01, AR 2 = .02), and 
being a private rather than a public institution (8 = .12, p < .05, AR 2 = .01). These findings 
suggest that baccalaureate institutions that adopt an extensive student assessment approach 
including the use of innovative assessment methods, provide broad internal access to student 
assessment information, have internal support for assessment, perceive state requirements as an 
important purpose of assessment, and are privately controlled will be more likely to use assessment 
information in decisions regarding faculty promotion and rewards. 

Conversely, viewing accreditation requirements as an important influence on assessment 
activities (8 = -. 17, p < .01 , AR 2 = .04), belonging to the North Central accrediting region (8 = - 
. 14, p < .01 , AR 2 = .02) and, to a lesser extent, having evaluated the assessment approach (8 = - 
.10, p < .05, AR 2 = .01) were negative predictors of faculty decisions being influenced by 
assessment information. This suggests that regionally-specific accreditation requirements 
concerning student assessment may differentially influence the likelihood of institutions using 
assessment information for faculty decisions in baccalaureate institutions. 

Faculty Impacts . This model explained 29% of the variance in faculty impacts from 
assessment in baccalaureate institutions (R 2 = .29). Six statistically significant predictors emerged, 
all but one of which were positive. The total number of assessment studies conducted by 
baccalaureate institutions was the strongest positive predictor, accounting for almost half of the 
explained variance in faculty impacts (8 = .24, p < .01, AR 2 = .14). Other positive predictors were 
the extent to which assessment information was incorporated in academic planning and review 
processes (8 = .22, p < .01, AR 2 = .07) and the number of institution-wide administrative and 
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governance activities promoting assessment (8 = .21 , p < .01 , AR 2 = .04). Having evaluated the 
assessment approach and having an institutional mission emphasis on undergraduate education 
and/or assessment were also positive predictors of faculty impacts from assessment but each were 
significant at the .05 level and contributed only 1% to explained variance in the model. Finally, 
using assessment information to make budget decisions among academic units was a negative 
predictor of positive faculty impacts from assessment; however, this index made only a minor 
contribution to the explained variance in the outcome measure (8 = -.1 1, p < .05, AR 2 = .01). 

This profile of predictors clearly suggests that baccalaureate institutions who analyze 
collected assessment information to understand the curricular and co-curricular experiences that 
affect students’ performance are more successful at enhancing faculty interest in teaching and 
encouraging their use of different teaching methods. Further, baccalaureate institutions that use 
assessment information in making academic planning and review decisions and that proffer 
institution-wide initiatives to promote assessment are more likely to document positive changes in 
faculty members’ teaching attitudes and practices. 

Student Impacts . The model regressing student impacts from assessment worked, 
moderately well for baccalaureate institutions, explaining 24% of the variance in this outcome 
measure (R 2 = .24). Five variables/indices were statistically significant predictors of student 
impacts. Incorporating assessment information into academic planning and review processes was 
the strongest of these, accounting for half the explained variance in the model (8 = .24, p < .01, 
AR 2 = .12). The number of assessment studies conducted (8 = .19, p < .01, AR 2 = .05) and 
policies providing professional development on assessment for student affairs personnel (8 = .18, 
p < .01, AR 2 = .03) were also positive predictors, as to a lesser extent was having evaluated the 
assessment approach (8 = .13,/? < .05, AR 2 = .01). Having a formal centralized institutional 
policy on student assessment was the only significant negative predictor of student impacts. 
However, its contribution to the explained variance in the model was very small (8 = -. 12, p < .05, 
AR 2 = .01). 
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Together these findings suggest that baccalaureate institutions that invest institutional 
resources into analyzing collected assessment information, include this information in processes 
concerning academic planning and review, and promote professional development on assessment 
among student affairs personnel will be more likely to achieve positive impacts on student 
performance from their assessment activities. 

External Impacts . The regression analysis of external impacts from assessment explained 
22% of the variance in this outcome measure for baccalaureate institutions (R 2 = .22). Seven 
statistically significant predictors emerged. Positive predictors included incorporating assessment 
information into academic planning and review processes (B = .16, p < .01, AR 2 = .10), having 
evaluated the assessment approach (B = .16,p < .01, AR 2 = .03), linking assessment to faculty 
evaluation policies (B = .14, p < .01, AR 2 = .03), collecting data on students’ cognitive 
competencies (B = .17, p < .01, AR 2 = .02), and having a mission emphasis on undergraduate 
education and assessment (B = ,1 1, p < .05, AR 2 = .01). These findings suggest, once again, the 
importance of including assessment information in academic planning and review processes. It 
may be that baccalaureate institutions that evaluate. their. assessment approach, include assessment- 
related criteria in faculty evaluation decisions and collect information on students’ cognitive 
competencies are better able to communicate institutional improvements stemming from assessment 
to their external constituents. 

Two significant negative predictors of external impacts from assessment were perceiving 
regional accreditation requirements as an influence on assessment activities (B = -.12, p < .05, AR 2 
= .02) and having an institution- wide group for assessment planning (B = -.12, p < .05, AR 2 = 
.01). Both predictors were at the .05 level of significance and accounted for a very small 
proportion of explained variance in the model. Speculation concerning their negative relationship 
to achieving positive external impacts from assessment would be premature at this time, but further 
consideration in future research seems warranted. 

Summary . Using R 2 values as the basis, the model for academic decisions worked best at 
predicting assessment uses and outcomes in baccalaureate institutions, followed by the model for 
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faculty decisions. Profiles of significant predictors varied across the five outcome measures. This 
was most pronounced for the model predicting the use of assessment information in faculty 
decisions. Unlike the other models, most of the significant predictors of faculty decision uses of 
assessment information came from the domains of external influences, internal characteristics and 
institutional approach to assessment while only one predictor was related to assessment 
management policies and practices. Three variables/indices appeared as important predictors in 
three or more models. The extent to which baccalaureate institutions incorporate assessment 
information into academic planning and review processes, total number of assessment studies 
conducted and having evaluated the assessment approach were consistently important aspects of 
assessment-related uses and impacts. Presumably, baccalaureate institutions that analyze collected 
assessment information, include this information in academic planning and review efforts, and 
evaluate the effectiveness of their assessment approach are more likely to shape institutional 
decisions in ways that promote institutional, faculty and student performance. 



11.3.3 Regression Results for Master’s Institutions 

Table 1 1 .5 presents the statistically significant predictors for each of the five regression 
analyses conducted for master’s institutions. 

Academic Decisions . The regression analysis predicting the use of assessment information 
in academic decisions worked quite well for master’s institutions, accounting for 49% of the 
variance in this outcome measure (R 2 = .49). Twelve variables were statistically significant 
predictors. The strongest of these, in terms of its contribution to the explained variance, was 
having policies that promoted student involvement in assessment activities (6 = .1 1, p < .05, AR 2 
= .18). The number of assessment studies conducted was also a strong positive predictor (6 =.21, 
p < .01, AR 2 = .11) as were conducting assessment for internal purposes (6 = .21, p < .01, AR 2 = 
.05), having professional development policies on assessment for student affairs personnel (6 = 
.15, p < .01, AR 2 = .05), having evaluated the assessment approach (6 = .19, p < .01, AR 2 = 
.03), an institutional mission emphasis on undergraduate education and assessment (B = .12,p < 
.01, AR 2 = .02), and linking assessment to faculty evaluation policies (6 = .09, p < .05, AR 2 = 
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Table 11.5 External and Internal Influences on Institutional Uses and Impacts of Student 
Assessment for Master’s Institutions (N=315) 



Institutional Uses and Impacts of Student Assessment 
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.11 




.24** .12 
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Assessment Management 














Policies & Practices 
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Budget decisions 






.18** .04 




.14** .02 




Access to information 


.09* 


.01 










Distribution of reports 


.09* 


.01 




.15** .03 




.20** .09 


Student involvement 


.11* 


.18 










Professional development 


.10* 


.01 


.16** .02 


.14* .02 






Student affairs 


.15** 


.05 










Faculty evaluation 


.09* 


.02 
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*p < .05; **p < .01 



.02). Collecting data on students’ cognitive competencies, providing professional development on 
assessment for academic administrators and faculty, breadth of access to assessment information 
on individual students, and distribution of assessment reports were also positive predictors of 
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using assessment information to make academic decisions but each contributed only 1 % to the 
explained variance in the model. The number of assessment instruments used was the sole 
statistically significant negative predictor but it, too, accounted for only 1% of the variance in 
academic decision uses of assessment (B = -.13, p < .05, AR 2 = .01). 

Assessment management policies and practices, particularly those encouraging student 
involvement in assessment, figure prominently as predictors of this use of assessment information 
in master’s institutions. In addition, the extent to which these institutions analyze the relationship 
between students’ experiences and their performance and view internal improvement as an 
important purpose of assessment are strong positive correlates of this outcome measure. 
Conversely, external influences and institutional characteristics do not appear to shape this 
assessment use. 

Faculty Decisions . Compared to the previous analysis, the model predicting the use of 
assessment information in faculty decisions did not work as well for master’s institutions; only 
20% of the variance in this outcome measure was explained (R 2 = .20). This comparatively poor 
model fit is partly attributable to the limited extent of this use of assessment information among 
master’ s institutions (see Table 7.2). 

Eight variables emerged as statistically significant predictors. Three of the strongest of 
these came from the domain of assessment management policies and practices: incorporating 
assessment information into academic planning and review processes (B = . 19, p < .01 , AR 2 = 

.06), using assessment information to make decisions regarding resource allocations to academic 
units (B = . 1 8, p < .01 , AR 2 = .04), and professional development policies on assessment for 
academic administrators and faculty (B = . 16, p < .01 , AR 2 = .02). This suggests that master’s 
institutions that link assessment information to their academic planning and budget processes and 
that provide professional development on assessment for academic affairs personnel and faculty are 
also more likely to connect assessment with faculty evaluation decisions such as salaiy and 
promotion. 
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The external influences domain provided three positive predictors: having state 
requirements regarding common student performance indicators or outcomes (B = .17, p < .01, 

AR 2 = .02), belonging to the Southern accrediting region (B = .13, p < .05, AR 2 = .01), and 
perceiving accreditation requirements as an important influence on assessment efforts (B = .\2,p < 
.05, AR 2 = .01). This implies that state and accrediting region assessment initiatives affect the 
likelihood of master’s institutions using assessment information for faculty decisions. 

Finally, having a formal centralized institutional policy on student assessment was a weak 
positive predictor while being a private institution was the only significant negative predictor (B = 
-.12, p < .05, AR 2 = .02). Institutional approach variables were not significantly related to this 
assessment use. 

Faculty Impacts . In this model, five positive statistically significant predictors explained 
25% of the variance in faculty impacts from assessment (R 2 = .25). The strongest of these was the 
total number of assessment studies conducted (B = .24, p < .01, AR 2 = .12) followed in turn by 
having a formal centralized institutional policy on assessment (B = .14, p < .01, AR 2 = .06), 
conducting an evaluation of the assessment approach (B = .16, p < .01, AR 2 = .03), the breadth of 
distribution of assessment reports (B = .15, p < .01, AR 2 = .03) and the provision of professional 
development on assessment for academic administrators and faculty (B = .14, p < .05, AR 2 = .02). 
This profile of predictors suggests that master’s institutions that adopt a standardized approach to 
assessment information collection, analyze and distribute this information, provide related 
professional development, and evaluate their assessment approaches are more likely to document 
associated positive impacts on faculty. 

Student Impacts . The model regressing student impacts from assessment explained 22% of 
the variance in this outcome measure (R 2 = .22). Five variables/indices emerged as positive 
statistically significant predictors of student impacts: one predictor each from the domains of 
institutional approach, external influences, and organizational and administrative support, and two 
predictors from assessment management policies and practices. The number of assessment studies 
conducted accounted for more than half of the explained variance in the model (B = .27, p < .01, 
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AR 2 = .13). This finding supports the importance of these institutions not only collecting but also 
analyzing assessment information if assessment efforts are to promote improved student 
performance. Incorporating assessment information into academic planning and review processes 
was the next strongest predictor (8 = .16 ,p< .01, AR 2 = .05). Belonging to the Southern 
accrediting region accounted for 2% of the variance in this outcome measure (8 = .16, p < .01, 

AR 2 = .02) as did the existence of a formal centralized institutional policy on assessment (8 = .13, 
p < .05, AR 2 = .02) and using assessment information in resource allocation decisions to academic 
units (8 = .14, p < .01, AR 2 = .02). These results suggest that master’s institutions that have 
adopted a centralized internal approach to assessment, have established formal linkages between 
assessment and their academic planning and resource allocation processes, and are in the Southern 
accrediting region are more likely to document positive changes in their students’ performance that 
are attributable to assessment efforts. 

External Impacts . The model predicting positive external impacts from assessment in 
master’s institutions accounted for 23% of the variance in this outcome measure (R 2 = .23). There 
were six positive statistically significant predictors distributed across the five conceptual domains 
of influences. In descending order of explained variance they were: distribution of assessment 
reports (8 = .20, p < .01, AR 2 = .09), conducting assessment for state purposes (8 = . 16 , p < . 01 , 
AR 2 = .04), number of assessment studies conducted (8 = .13, p < .05, AR 2 = .03), total 
institutional enrollment (8 = . 13, p < .05, AR 2 = .02), conducting assessment for internal purposes 
(8 *s .1 \,p < .05, AR 2 = .02), and collecting data on students’ post-college performance (8 = .12, 
p < .05, AR 2 = .01). In many ways, this profile of predictors makes intuitive sense. That is, the 
greater the extent to which master’s institutions perceive an external purpose such as state 
requirements as motivating their assessment efforts, collect information regarding the post- 
enrollment performance of former students, and produce and distribute reports of assessment 
results, the more likely they are to achieve positive external impacts from their assessment 
activities. 
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Summary . Of all five models analyzed, the model predicting the use of assessment 
information in academic decisions produced the best fit. The pattern of significant predictors 



varied for each outcome measure. Overall, the domains of internal influences exceeded those of 
external influences and institutional characteristics as significant predictors of assessment uses and 
impacts. The main exception to this general pattern was the model predicting the use of assessment 
information in faculty decisions. Regression results suggest this use of assessment information is 
more likely to stem from external than from internal motives. Six variables/indices emerged as 
important predictors in several regression models: number of assessment studies conducted, 
distribution of assessment reports, professional development on assessment for academic 
administrators and faculty, formal centralized institutional policy on assessment, and the 
incorporation of assessment information in academic planning and resource allocation decisions. 
Together these results imply that master’s institutions characterized by extensive efforts to integrate 
assessment practices into ongoing institutional processes and that analyze and broadly 
communicate assessment results are more likely to use and be positively affected by assessment 
information. 

11.3.4 Regression Results for Doctoral and Research Institutions 

Table 1 1.6 presents the statistically significant predictors for each of the five regression 
analyses conducted for doctoral and research institutions. 

Academic Decisions . The model regressing the use of assessment information in academic 
decisions worked well for doctoral and research institutions explaining 47% of the variance in this 
outcome measure (R 2 = .47). Eight variables/indices were statistically significant predictors. The 
strongest predictors, in terms of explained variance, came from the domain of assessment 
management policies and practices. These were providing professional development on 
assessment for academic administrators and faculty (6 = .28, p < .01, AR 2 = .19), including 
assessment-related criteria in policies for faculty evaluation (6 = .26, p < .01, AR 2 = .11) and 
policies promoting student involvement in assessment (6 = .17, p < .05, AR 2 = .03). The number 
of assessment studies conducted was also a strong predictor of this outcome measure (6 = .22, p < 
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Table 11.6 External and Internal Influences on Institutional Uses and Impacts of Student 
Assessment for Doctoral and Research Institutions (N=145) 
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Policies & Practices 
Access to information 


.14* 


.02 






.18* .03 




Distribution of reports 
Student involvement 


.17* 


.03 


.21* .04 


.19* .06 


.16* .05 




Professional development 


.28** 


.19 




.25** .07 






Faculty evaluation 


.26** 


.11 


n/inc 




.42** .22 


.25** .05 



< .05; **p < .01 



.01, AR 2 = .07). The remaining significant predictors made comparatively smaller contributions to 
the explained variance in this model. Two of these were positive predictors from the organizational 
and administrative support for assessment domain: conducting assessment for internal purposes (8 
= . 16, p < .05, AR 2 = .02) and the degree of administrative and faculty support for assessment (8 
= A4,p< .05, AR 2 = .02). Two were negative: being a privately controlled institution (6 = -.15, 
p < .05, AR 2 = .02) and having an emphasis on undergraduate education and assessment in the 
institutional mission (8 = -.16, p < .05, AR 2 = .02). 

The relationship among the positive predictors and using assessment information in 
academic decisions appears relatively straightforward. Doctoral and research institutions that 
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support assessment efforts by integrating them within management policies and practices, that 
study the relationship between students’ institutional experiences and performance, and that have 
achieved a positive degree of support for assessment from administrators and faculty are more 
likely to use assessment information to shape academic decision making. The negative relationship 
between academic decision uses of assessment and institutional control suggests that private 
doctoral and research institutions do not use assessment information in academic decision making 
to the same extent as their public counterparts. Since this research focused on undergraduate 
student assessment, it is likely that information collected from these assessment efforts may have 
little effect on these primarily graduate education-oriented institutions’ academic decisions. This 
may explain the negative relationship between mission emphasis and academic decision uses of 
assessment information. 

Faculty Decisions . The use of assessment information in faculty decisions was not well 
predicted by this model (R 2 = .04). Only one index emerged as a statistically significant predictor. 
The extent of policies promoting student involvement in assessment explained 4% of the variance 
in this outcome measure (B = .21, p < .05, AR 2 = .04). This result is partly attributable to the low 
frequency of this assessment use among doctoral and research institutions (see Table 7.2). 

Drawing further conclusions regarding the predictors of this assessment use among these 
institutions is inappropriate without further analysis. 

Faculty Impacts . In contrast to the analysis discussed above, the model regressing faculty 
impacts from assessment accounted for 34% of the variance in this outcome measure, the best fit 
for this model among all types of institutions (R 2 = .34). Four variables/indices were statistically 
significant positive predictors of this measure. The strongest of these was the number of 
institution-wide ad minis trative and governance activities undertaken to promote student assessment 
(B = .26, p < .01, AR 2 = .12). This was followed by the provision of professional development 
on assessment for academic administrators and faculty (B = .25, p < .01, AR 2 = .07), breadth of 
distribution of assessment reports (B = .19,p < .05, AR 2 = .06) and collection of data on students’ 
post-college competencies (B = .17, p < .05, AR 2 = .05). This suggests that doctoral and research 
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institutions that have visible administrative/leadership support for assessment, that develop policies 
to encourage student involvement and enhance administrators’ and faculty members’ knowledge of 
assessment-related issues, and that collect data on former students’ employment, education and 
satisfaction are more likely to observe positive changes in faculty members’ undergraduate teaching 
attitudes and practices. Conversely, being a private rather than a public institution was the only 
significant negative predictor of positive faculty impacts from assessment (B = -.15, p < .05, AR 2 
= .05). 

Student Impacts . The model predicting student impacts from assessment also worked well 
for doctoral and research institutions. Five statistically significant predictors emerged accounting 
for 36% of the variance in this outcome measure (R 2 = .36). Of these predictors, using 
assessment-related criteria in faculty evaluation policies was clearly the strongest (B = .42, p < .01, 
AR 2 = .22). This finding underlines the potential importance of this management policy as a lever 
to enhance faculty interest in issues related to undergraduate teaching and assessment. Other 
significant positive predictors included: the breadth of distribution of assessment reports (B = .16, 
p < .05, AR 2 = .05), internal access to assessment information concerning individual students (B = 
AS,p < .05, AR 2 = .03), and number of administrative and governance activities promoting 
student assessment (B = .20, p < .01, AR 2 = .03). These results suggest doctoral and research 
institutions that provide regular access to student assessment information and demonstrate 
leadership support for assessment through institution-wide assessment initiatives are more likely to 
document positive impacts from assessment on students’ achievement within and beyond the 
institution. 

Again, being a privately controlled institution was the only significant negative predictor of 
this outcome measure (B = -.20, p < .01, AR 2 = .03). Private institutions are not subject to the 
same degree of scrutiny from state officials as public institutions. Thus they may feel less pressure 
to document student impacts from assessment. 

External Impacts . The regression analysis for external impacts from assessment accounted 
for 26% of the variance in this outcome measure (R 2 = .26). Three variables/indices were 
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statistically significant predictors: two were positive and one was negative. The number of 
assessment studies conducted contributed the largest proportion of the explained variance in the 
model (B = .35, p < .01, AR 2 =.16) followed by the extent to which assessment-related criteria 
were used in faculty evaluation policies (B = .25, p < .01, AR 2 = .05). Being a privately controlled 
institution was a negative predictor of positive external impacts from assessment (B = -.23, p < 

.01, AR 2 = .05). 

These results suggest that doctoral and research institutions that analyze the relationship 
between their students’ institutional experiences and academic performance, and link assessment 
involvement or performance data with faculty evaluation and rewards are more likely to enhance 
their performance on external indicators of performance such as state funding allocation, student 
application and acceptance rates and institutional reputation. Compared to their public 
counterparts, private doctoral and research institutions often enjoy higher institutional prestige, 
make less use of state funds, and are able to be more selective in their admissions processes. 

Thus, private control is less likely to be associated with gains in these external impacts from 
assessment. 

Summary . As was the case for other types of institutions, the use of assessment 
information in academic decisions was the outcome measure best predicted by this model for 
doctoral and research institutions. The models predicting faculty and student impacts from 
assessment also worked comparatively well for doctoral and research institutions while the model 
predicting the use of assessment information in faculty decisions did not fit well. 

The profile of statistically significant predictors varied for each of the five outcome 
measures but some common predictors were also observed. Variables from the external influences 
domain did not emerge as significant predictors in any model suggesting that the uses and impacts 
of assessment among doctoral and research institutions are generally unaffected by state-level and 
regional accreditation initiatives on assessment. Private institutional control was consistently 
negatively associated with assessment uses and impacts. The inclusion of assessment-related 
criteria in faculty evaluation policies was a strong positive predictor in three of the four analyses in 



ERIC 



296 




which it was included. This suggests faculty evaluation policies may be a powerful institutional 
mechanism for promoting assessment uses and impacts. Finally, the provision of professional 
development opportunities on assessment for academic administrators and faculty, and number of 
assessment studies also appeared as strong positive predictors of using and achieving observable 
impacts from assessment information in doctoral and research institutions. 

11.4 External and Internal Influences by Institutional Uses and Impacts of Assessment 
The previous sections in this chapter have considered the results of regression analyses of 
the five assessment outcome measures for all institutional respondents and separately for each of 
four types of institutions. In this section, we examine similarities and differences in the patterns of 
statistically significant predictors for each outcome measure across the four types of institutions. 

11.4.1 Predictors of Uses in Academic Decisions 

Table 1 1.7 presents the statistically significant predictors of the academic decisions factor 
index for the four types of institutions considered in our analyses: associate of arts, baccalaureate, 
master’s, and doctoral and research. Explained variance in this factor was quite high for all types 
of institutions, ranging from 40% for baccalaureate institutions to 49% for master’s institutions. 
Both common and distinctive patterns of predictors of academic decision uses of assessment 
information were evident in the regression results for these four types of institutions. 

For all types of institutions, the domains of external influences aiid institutional 
characteristics had little significant relationship to this use of assessment information. Conversely, 
the domains of institutional approach and of organizational and administrative support for 
assessment provided several statistically significant predictors. Two variables from these domains 
were consistently strong predictors of institutional use of assessment data to inform academic 
decisions: the number of studies conducted to analyze the relationship between students’ 
institutional experiences and their performance, and the importance rating given to internal 
improvement as a purpose for engaging in student assessment efforts was a strong predictor. 
These results reinforce the importance of institutions not only collecting assessment data but also 



Table 11.7 External and Internal Influences on Use of Student Assessment for Academic 
Decisions by Institutional Type 



Institutional Type 



Associate 
of Arts 
N=548 



Baccalaureate 

N=316 



Master’s 

N=315 



Doctoral & 
Research 
N=145 



.41** 



.40** 

Beta AR 2 



.49** 



.47** 



R 2 

External Influences 

Southern 

Institutional Characteristics 

Control (l=public, 2=private) 

Institutional A pproach 

Cognitive assessment 
Post-college assessment 
Number of instruments 
Student-centered methods 
Number of studies 
Organizati onal & Adm in i strat iv e 

Support 

Mission emphasis 
Internal purposes 
Administrator & faculty support 
Conducted evaluation 
Assessment Management 
Po licie s fo JQracti ces 
Computer support 
Access to information 
Distribution of reports 
Student involvement 
Professional development 
Student affairs 

Faculty evaluation 

*p < .05; **p < .01 



. 12 ** .02 



.10* 


.09 






.08* 


.01 


.13** 


.02 


.13** 


.02 






.16** 


.05 


.16** 


.03 


.12** 


.03 


.19** 


.09 



.10** 


.01 






.14** 


.02 






.08* 


.01 


.16** 


.03 






.12* 


.17 


.16** 


.14 


.12* 


.01 


.10** 


.01 


.18** 


.05 



Beta 


M? 


Beta 


AEl 






-.15* 


.02 


.11* 


.01 






-.13** 


.01 






.21** 


.11 


.22** 


.07 


.12** 


.02 


-.16* 


.02 


.21** 


.05 


.16* 


.02 






.14* 


.02 


* 

# 

ON 


.03 







.09* 


.01 






.09* 


.01 






.11* 


.18 


.17* 


.03 


.10* 


.01 


.28** 


.19 


.15** 


.05 






.09* 


.02 


.26** 


.11 



making efforts to systematically analyze this data. Further, they provide some support for 
scholars’ contentions that assessment undertaken for internal rather than external purposes will be 
more likely to produce institutional improvements (Aper et al., 1990). The domain of assessment 
management policies and practices emerged as the most important influence on institutions’ use of 
assessment information in academic decisions. For all institutional types, variables in this domain 
accounted for the greatest proportion of explained variance. This finding supports the- views of 
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scholars such as Ewell (1984, 1987a, 1987b, 1987c, 1988a, 1988b, 1997) and Banta (Banta & 
Associates, 1993; Banta et al., 1996) who assert such policies and practices are powerful 
institutional levers for promoting student assessment in colleges and universities. 

However, regression results also revealed differences in predictors of academic decision 
uses across types of institutions. For associate of arts institutions only, the extent to which 
cognitive performance data was collected was a strong predictor of academic decision uses of 
assessment information. While the domain of assessment management policies and practices was 
an important source of influence for all institutions, the specific policies and practices that most 
effectively promote the use of assessment information in academic decision making varied with 
institutional type. For associate of arts institutions, professional development policies directed at 
student affairs personnel were most influential. In master’s institutions, policies encouraging 
student involvement and providing professional development for student affairs personnel 
accounted for the most variance in this outcome measure. In baccalaureate, doctoral and research 
institutions, a broader array of policies emerged. Policies providing professional development for 
academic administrators and faculty members, policies linking assessment to faculty evaluation, 
and to a lesser extent, policies encouraging student involvement were strong predictors in this 
domain. These findings suggest that internal responsibility for student assessment varies by 
institutional type. Thus policies intended to promote student assessment must target 
correspondingly different internal constituent groups. 

11.4.2 Predictors of Uses in Faculty Decisions 

Table 1 1 .8 presents the statistically significant predictors of the faculty decisions factor 
index for associate of arts, baccalaureate, master's, and doctoral and research institutions. 

The ability of this model to predict institutional use of assessment information for faculty 
decisions varied considerably by institutional type. Explained variance ranged from 4% in doctoral 
and research institutions to 31% in baccalaureate institutions. Differences in this institutional use 
of assessment information may be reflective of corresponding differences in faculty autonomy in 
specific types of institutions. Overall, institutional characteristics and variables concerning 
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Table 11.8 External and Internal Influences on Use of Student Assessment for Faculty 
Decisions by Institutional Type 



Institutional Type 





Associate 










Doctoral & 




of Arts 


Baccalaureate 


Master’s 


Research 




N =548 


N =316 


N =315 


N=145 


R 2 


.12** 


.31 


** 


.20 


* * 


.04** 




Beta AR 2 


ism 


AR 2 


Beta 


AR 2 


Beta AR 2 


External Influences 
North Central 
Southern 


-. 11 * .02 


-. 14 ** 


.02 


. 13 * 


.01 




State approach 
Accreditation purposes 
State purposes 


-. 09 * .01 


. 21 ** 


.02 


. 17 ** 


.02 




Accrediting influence 
Institutional Characteristics 




-. 17 ** 


.04 


. 12 * 


.01 




Control (l=public, 2=private) 
Institutional Approach 




. 12 * 


.01 


-. 13 * 


.02 




Cognitive assessment 




. 17 ** 


.03 








Student-centered methods 




. 17 ** 


.02 








External methods 
Number of studies 


. 16 ** .03 


. 17 ** 


.06 








Organizational & Administrative 
Support 

Administrator & faculty support 
Formal centralized policy 
Institution- wide group 


-. 09 * .01 


. 13 ** 


.02 


. 14 * 


.01 




Conducted evaluation 
Assessment Manaeement 
Policies & Practices 
Academic planning & review 


. 15 ** .05 


-. 10 * 


.01 


. 19 ** 


.06 




Budget decisions 
Access to information 




. 14 ** 


.09 


. 18 ** 


.04 




Student involvement 
Professional development 


. 12 ** .01 






. 16 ** 


.02 


. 21 * .04 



*p < . 05 ; **p < .01 



organizational and administrative support for assessment had little significant relationship to this 
use of assessment information. The domain of assessment management policies and practices 
contributed the most and strongest predictors of this dependent measure. Beyond these 
similarities, patterns of statistically significant predictors differed for each type of institution. 
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External influences were most pronounced for baccalaureate institutions, played a weaker 
role in associate of arts and master’s institutions, and did not contribute significantly to faculty 
decision uses of assessment information in doctoral and research institutions. The institutional 
approach domain made the greatest contribution to faculty decisions uses of assessment 
information for baccalaureate institutions, was a weaker source of influence in associate of arts 
institutions, and did not contribute significantly to faculty decisions uses in master’s, doctoral and 
research institutions. Among baccalaureates, using assessment methods that involve external 
participants and collecting data on students’ cognitive competencies were strong predictors. 
Considering assessment management policies and practices, significant predictors in this domain 
were quite unique to each institutional type. Incorporating assessment information into academic 
planning and review processes was an important predictor of faculty decision uses of assessment 
information for associate of arts institutions. For baccalaureate institutions, provi din g internal 
constituents with access to student assessment information was a strong contributor to faculty 
decision uses. Among master’s institutions, incorporating assessment data into academic planning 
and review processes and linking assessment data to resource allocation decisions were strong 
predictors. Finally, having policies encouraging student involvement in assessment was the only 
strong predictor of using assessment information in faculty-related decisions for doctoral and 
research institutions. These findings support the important role of assessment management 
policies and practices in promoting the use of assessment information. They again suggest that the 
importance of specific types of policies and practices in promoting assessment varies by 
institutional type. 

11.4.3 Predictors of Faculty Impacts 

Table 1 1.9 presents the statistically significant predictors of the faculty impacts index for 
associate of arts, baccalaureate, master’s, and doctoral and research institutions. This model 
worked moderately well for all types of institutions. Explained variance in faculty impacts from 
assessment ranged from 25% to 34%. 
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Table 11.9 External and Internal Influences on Faculty Impacts of Student Assessment by 
Institutional Type 

Institutional Type 



Associate Doctoral & 

of Arts Baccalaureate Master’s Research 

N=548 N=316 N=315 N=145 



R 2 


.28 


* * 


.29 


* * 


.25 


* * 


.34 


* * 




Beta 


AR 2 


Beta 


AR 2 


Beta 


Ml 


Beta 


AR 2 


External Influences 
State approach 


-.09* 


.01 














Accreditation purposes 
Institutional Characteristics 
Control (l=public, 2=private) 
Institutional Approach 


-.12** 


.01 










-.15* 


.05 


Post-college assessment 
Student-centered methods 


.09* 


.01 










.17* 


.05 


Number of studies 


.14** 


.06 


.24** 


.14 


.24** 


.12 






Qrgaimtional & Administrative 
Support 

Mission emphasis 






.10* 


.01 










Admin. & governance activities 
Formal centralized policy 


.17** 


.04 


.21** 


.04 


.14** 


.06 


.26** 


.12 


Conducted evaluation 


.08* 


.01 


.11* 


.01 


.16** 


.03 






Assessment Management 
Policies & Practices 
Academic planning & review 


.10* 


.02 


.22** 


.07 










Budget decisions 
Computer support 
Distribution of reports 


.09* 


.01 


-.11* 


.01 


.15** 


.03 


.19* 


.06 


Professional development 


.17** 


.11 






.14* 


.02 






Faculty evaluation 


.11** 


.01 










.25** 


.07 



*p < .05; **p < .01 



Overall, variables from the domains of external influences and institutional characteristics 
were not important sources of influence on faculty impacts attributed to institutions’ assessment 
activities. The index for total number of assessment studies conducted emerged as a very strong 
positive predictor of faculty impacts in all institutional types except doctoral and research 
institutions. Among doctoral and research institutions, collecting data on students’ post-college 
competencies (e.g., professional outcomes, further education, satisfaction) was a strong predictor 
of faculty impacts. This may reflect the tendency of faculty in graduate-level institutions to place 
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more responsibility on students for their own performance and thus to be less influenced by 
information concerning the relationship of students’ institutional experiences to their performance. 
The number of institution-wide administrative and governance activities intended to promote 
assessment was a strong predictor of faculty impacts for all but master’s institutions. Presumably, 
these initiatives demonstrate that leadership support for assessment and broad internal participation 
in decision making concerning assessment are positively related to achieving faculty impacts from 
assessment. The domain of assessment management policies and practices was again a 
consistently important influence on assessment-related outcomes. However, the strength of 
individual predictors within this domain varied by institutional type. 

For associate of arts institutions, providing professional development for faculty and 
academic administrators was the strongest predictor of faculty impacts. For baccalaureate 
institutions, incorporating assessment information into academic planning and review processes 
was a significant predictor of faculty impacts. Distributing reports of assessment results was an 
important predictor among master’s, doctoral and research institutions, while linking assessment to 
faculty evaluation was a strong predictor among doctoral and research institutions. These results 
suggest that profiles of effective assessment management policies and practices for promoting 
student assessment must be uniquely crafted for specific types of institutions. 

1 1.4.4 Predictors of Student Impacts 

Table 1 1. 10 presents the statistically significant predictors of the student impacts index for 
associate of arts, baccalaureate, master's, and doctoral and research institutions. This model 
worked best for doctoral and research institutions, explaining 36% of the variance. It did not work 
as well for the other types of institutions, explaining from 22% to 24% of the variance in student 
impacts from assessment. 

For all types of institutions, the domains of external influences, institutional characteristics 
and administrative and governance support for assessment were of minor importance as predictors 
of positive student impacts from assessment. The assessment policies and practices domain 
provided the most significant predictors of this dependent measure. For all but doctoral and 
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Table 11.10 External and Internal Influences on Student Impacts of Student Assessment by 
Institutional Type 



Institutional Type 




Associate 








Doctoral & 




of Arts 


Baccalaureate 


Master’s 


Research 




N=548 


N=316 


N=315 


N=145 


R 2 


.22** 


.24** 


.22 


* * 


.36** 




Beta AR 2 


Beta AR 2 


Beta 


AR 2 


Beta AR 2 


External Influences 
Southern 






.16** 


.02 




Institutional Characteristics 
Control (l=public, 2=private) 
Institutional Approach 
Number of studies 
Organizational & Administrative 
Support 


.14** .03 


.19** .05 


.27** 


.13 


-.20** .03 


Admin. & governance activities 
Formal centralized policy 




-.12* .01 


.13* 


.02 


.20** .03 


Conducted evaluation 




.13* .01 








Assessment Management 
Policies & Practices 












Academic planning & review 


.21** .13 


.24** .12 


.16** 


.05 




Budget decisions 
Computer support 
Access to information 


.15** .04 




.14** 


.02 


.18* .03 


Distribution of reports 
Student affairs 


.13** .02 


.18** .03 






.16* .05 


Faculty evaluation 


.11** .01 








.42** .22 



*p < .05; **p < .01 



research institutions, linking assessment data with academic planning and review processes, and 
conducting studies of assessment data were the most important influences on the likelihood of 
assessment leading to improved student performance. The comparative unimportance of these 
predictors for doctoral and research institutions may be due to these institutions’ focus on graduate 
rather than undergraduate education. 

In addition to the influences discussed above, computer support for assessment information 
was an important predictor of student impacts in associate of arts institutions. Compared to other 
types of institutions, associate of arts colleges tend to have a student body that is more diverse in 
terms of sociodemographic profile, academic preparedness and enrollment patterns. For these 



institutions, having the technical capacity to collect and integrate assessment information may be a 
particularly important precursor to being able to analyze and apply assessment results for the 
betterment of students’ performance. Among baccalaureate institutions, providing professional 
development on assessment for student affairs personnel was a strong predictor of student impacts. 
This suggests that student affairs personnel play an important role in utilizing assessment results in 
these institutions. 

Compared to other institutional types, the predictors of student impacts from assessment 
among doctoral and research institutions were distinctive. Including assessment-related criteria in 
faculty evaluation policies was the strongest predictor by far, accounting for two-thirds of the 
explained variance. Distribution of assessment reports, internal access to assessment information 
on individual students, and the number of administrative and governance activities promoting 
assessment were also strong positive predictors of student impacts. One institutional characteristic, 
private control, was a significant negative predictor. The apparent importance of linking 
assessment to faculty rewards in order to achieve positive student impacts from assessment may be 
attributable to the typical emphasis on research rather than teaching in these graduate-oriented 
institutions. 

11.4.5 Predictors of External Impacts 

Table 11.11 presents the statistically significant predictors of the external impacts index for 
associate of arts, baccalaureate, master's, and doctoral and research institutions. This model 
worked moderately well for all types of institutions. It explained from 22% to 26% of the variance 
in external impacts from assessment. 

For most types of institutions, variables from the domains of external influences, 
institutional characteristics, institutional approach to assessment, and organizational and 
administrative support for assessment were of minor importance as predictors of external impacts 
of assessment. The assessment management policies and practices domain was the strongest 
source of influence on external impacts for all types of institutions except doctoral and research. 
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Table 11.11 External and Internal Influences on External Impacts of Student Assessment by 
Institutional Type 

Institutional Type 

Associate Doctoral & 

of Arts Baccalaureate Master’s Research 

N=548 N=316 N=315 N=145 

. 23 ** . 22 ** . 23 ** . 26 ** 



R 2 

External Influences 
State purposes 
Accrediting influence 
Institutional Characteristics 
Control (l=public, 2=private) 
Enrollment 
Institutional Approach 
Cognitive assessment 
Post-college assessment 
Number of instruments 
Number of studies 
Organizational & Administrative 
Support v 
Mission emphasis 
Internal purposes 
Admin. & governance activities 
Institution- wide group . 
Conducted evaluation: :.:. 
Assessment Management 
Policies & Practices 
Academic planning & review 
Budget decisions 
Computer support 
Distribution of reports 
Student involvement 
Faculty evaluation 
*p < .05; **p < .01 



Beta AR 2 Beta AR 2 



-. 12 * .02 



.17** .02 

.09* .01 

. 10 * .01 



.11* .01 

.12** .03 

-. 12 * .01 
.16** .03 



.13** .10 .16** .10 

.08* .01 
.18** .05 

. 10 * .01 

.13** .02 .14* .03 



Beta AR 2 Beta AR 2 
.16** .04 

-.23** .05 

.13* .02 

.12* .01 

.13* .03 

.35** .16 

.11* .02 



.20** .09 

.25** .05 



The strength of specific variables as predictors in this model varied considerably by institutional 
type. 

Among associate of arts institutions, the strongest predictors were incorporating 
assessment information into academic planning and review processes, computer support for 
assessment information, and the number of administrative and governance activities promoting 
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assessment. In baccalaureate institutions, the strongest predictors of external impacts were 
incorporating assessment information into academic planning and review processes followed by 
including assessment-related criteria in faculty evaluation policies and having evaluated the 
assessment process. The strongest predictors of external impacts from assessment among master’s 
institutions were the distribution of assessment reports followed by conducting assessment in order 
to meet state requirements and the number of assessment instruments used. There were just three 
statistically significant predictors of external assessment impacts in doctoral and research 
institutions. The strongest was the total number of assessment studies conducted. Including 
assessment-related criteria in faculty evaluation policies was a positive but comparatively weaker 
predictor while being a private institution was a negative predictor of external impacts. 

The importance of these findings must be tempered by the modest amounts of variance 
explained by this model. However, they still reinforce the need to customize support for student 
assessment for distinct types of institutions. For example, these findings suggest the importance 
of promoting student assessment through multiple means (technological support, leadership, and 
institutional processes) if assessment information is to have a positive impact on associate of arts 
institutions’ relationships with their external constituencies. For baccalaureate institutions, Unking 
assessment to academic planning processes is important Distributing assessment reports and 
conducting assessment to fulfill state requirements are related to positive external impacts among 
master’s institutions. More so than other types of institutions, doctoral and research universities 
appear to use results from assessment studies to promote their institution among external 
constituencies. 

11.4.6 Summary of Predictors of Assessment Use and Impact Measures bv Institutional Type 

The regression models were most effective for predicting the use of assessment information 
in academic decisions and least effective for predicting the use of assessment information in faculty 
decisions. Differences in explained variance for each model are partly attributable to corresponding 
differences in the frequency with which institutions reported these assessment uses and impacts. 
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Some general patterns of predictors of assessment uses and impacts were evident across 
types of institutions. Overall, internal influences were stronger predictors than external influences. 
Among internal domains, assessment management policies and practices contributed the most and 
strongest predictors of assessment uses and impacts while institutional characteristics contributed 
the fewest and weakest predictors. The domains of institutional approach to assessment, and 
organizational and administrative support were moderately important sources of influence. 

The following indices emerged as strong predictors of assessment use and impact measures 
across institutional types: incorporating assessment information into academic planning and review 
processes, total number of assessment studies conducted, number of administrative and 
governance activities promoting assessment, professional development on assessment for academic 
administrators and faculty, professional development on assessment for student affairs personnel, 
and including assessment-related criteria in faculty evaluation policies. Together, these findings 
give credence to forms of institutional support for student assessment advocated in the literature. 

However, there was also substantial variation observed in the patterns of statistically 
significant predictors across types of institutions. This clearly suggests that effective structures, 
strategies and processes of institutional support for student assessment will be configured 
differently for specific types of institutions. 
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12. Evolution Not Revolution: A Concluding Summary 



12. 1 Overview 

This survey, an Inventory of Institutional Support for Student Assessment, is the first 
comprehensive national survey of how higher education institutions approach and support 
undergraduate student assessment. After more than a decade of activity, it provides a base for 
examining the nature and extent of institutional efforts to make student assessment a common and 
widely used part of the academic management function at two-year and four-year colleges, and 
comprehensive and research universities. The results suggest a picture of an evolutionary — not 
revolutionary — pattern of adopting various student assessment approaches, of developing 
institutional mechanisms to support and promote assessment, and of using and monitoring the 
impact of assessment data on student and institutional performance. 

In general, the results suggest the following: While requirements for student assessment 
emanating from state agencies and institutional accrediting bodies have stimulated the initial 
adoption of assessment activities among institutions, they appear to have little influence on the 
likelihood of institutions using and achieving positive impacts from student assessment 
information. The majority of institutions report engaging in some student assessment activity. But 
most often they have adopted only one or two types of student assessment measures, conduct the 
assessment at only one point m time during students’ enrollment, and tend to use rather traditional, 
mostly quantitative methods of assessment. Institutions report moderate use of institution-wide 
governance, administrative and leadership activities intended to promote student assessment Their 
introduction of policies and practices designed to support the use of student assessment is mixed, 
relying most often on efforts in the areas of academic planning and student involvement and 
seldom in areas such as rewarding and evaluating faculty. To date, institutional efforts to use 
student assessment data are very limited. They are most likely to use assessment data to modify 
assessment plans or processes and least likely to report wide-scale use of student assessment data 
in faculty promotion and reward decisions. For the most part, they have not attempted to monitor 



the internal or external impacts of their assessment efforts. This general pattern of results varies by 
institutional type. Overall, the picture of institutional support for student assessment that emerges 
from our data is an evolutionary one: considerable adoption of some types of student assessment 
measures, some institutional effort to support and promote assessment, and very little attention to 
actually using the information generated or to monitor its impacts. 

12.2 The Study 

The study was based on an institutional survey which required institutions to complete an 
inventory of their undergraduate student assessment activity. A comprehensive literature review 
and synthesis of the organizational and administrative factors related to student assessment was 
used to create a conceptual framework to guide the study (see Figure 1.2) and to identify specific 
survey items. The framework identified five environments of institutional support for student 
assessment: external influences; institutional approach to student assessment; organizational and 
administrative support for student assessment; institutional context; and the institutional use of 
student assessment information in decision making and its impact on the institution. The 
organizational and administrative support environment was comprised of the domains of 
institutional support strategy for assessment, assessment leadership and governance patterns, 
assessment management policies and practices, evaluation of the student assessment process, and 
assessment culture and climate. The latter domain was not included in this institutional survey but 
will be examined in the intensive case studies conducted in the next phase of this research project 

The population of all higher education institutions offering associate or baccalaureate 
degrees was surveyed. Slightly more than 55% responded in time to be included in the analysis. 
The responses by each of five institutional types (associate of arts, baccalaureate, master’s, 
doctoral and research) were also high. 

The analysis that followed examined descriptive patterns of response to all items 
(frequencies, means, and standard deviations) for all respondents and by institutional type and 
control. A descriptive profile of the results was reported in chapters three to seven of this report. 
Factor analysis, creation of summative indices, and identification of key item variables allowed us 



to reduce the number of variables used for multivariate analysis. Bivariate and regression analysis 
allowed us to examine the relationship of variables in the five environments and to examine the 
relative influence of external influences, institutional approach to student assessment, institution- 
wide support patterns and assessment management policies and practices on the institutional use 
and impacts of student assessment information. These multivariate analyses were reported in 
chapters nine to eleven. 

12.3 Results 

While the preceding chapters provide extensive and detailed descriptive results in each of 
the five survey environments and examine the relationships among them, the following summary 
highlights the key findings. The presentation of these findings is consistent with the eight research 
questions (see chapter one) which guided the study. The findings for research questions one 
through four and six through eight are presented for all responding institutions. The findings 
related to research question five highlight the results by institutional type and control across all five 
environments of the conceptual framework. 

1 . What types of measures and approaches to student assessment have institutions adopted? 

• Institutions most often collect data on current students’ academic progress, basic college- 
readiness skills, and academic intentions and least often collect information on higher-order 
skills, affective development, and-former students’ civic or social roles. Their approaches 
emphasize the use of entry measures and easily quantifiable indicators of student progress and 
give less attention to more complex measures of students’ cognitive and affective domains. 
(Ch. 3) 

• There is evidence that institutions are beginning to collect student assessment information at 
more than one point in time during students’ involvement with the institution. (Ch. 3) 

• When standardized instruments are used, they are more likely to have been developed by the 
institution itself rather than provided by state or commercial sources. (Ch. 3) 




3ii 36 3 

1 I V 



• Overall, institutions make limited use of less traditional student assessment methods such as 
portfolios, capstone projects, observations of student performance, and interviews or focus 
groups with current students, employers or alumni. (Ch. 3) 

• Institutions do provide descriptive reports of student assessment information. Most often 
results are profiled on an institution-wide basis or disaggregated by academic program or 
department. However, few institutions conduct studies of the relationship between students’ 
institutional experiences and student performance. (Ch. 3) This is an important finding in view 
of the relationship of assessment studies to institutional use and impacts of assessment 
information. (Ch. 1 1) 

2. What is the nature of external influences for student assessment in postsecondary institutions? 

State Role 

• Institutions reporting the existence of a state plan for student assessment (half of respondents) 
most often perceive state requirements as having a positive influence on their assessment 
efforts. (Ch. 4) 

• Institutions reporting the existence of a state plan for student assessment report they are more 
often required by state officials to provide evidence of a student assessment plan than use of 
student assessment results. Student performance indicators, if required, are more likely to be 
state-mandated than institutionally-devised. (Ch. 4) 

• Half of institutions reporting the existence of a state plan for student assessment have had their 
assessment efforts reviewed, most often by state-level officials. Reviews have focused on the 
assessment process itself rather than on reports of student performance. (Ch. 4) 

Accreditation Role 

• Most institutions have undergone a regional accreditation review requiring student assessment. 
(Ch. 4) 

• The majority perceive regional accreditation requirements as having had a positive influence on 
their assessment activities. (Ch. 4) 
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• Institutions report that regional accreditors most often require institutions to provide evidence 
of an assessment plan and least often require evidence of assessment results. Compared to 
state officials, regional accreditors more often require evidence of intended or actual 
institutional use of assessment information. (Ch. 4) 

Other External Sources of Support for Assessment 

• Few institutions have received grants from external sources to improve their student 
assessment practices (Ch. 4) 

• The majority of institutions have used conferences, publications or research reports on student 
assessment provided by a variety of postsecondary organizations. Professional associations 
and regional accrediting associations are the major providers of these assessment support 
services. (Ch. 4) 

3 . What organizational and administrative support patterns for student assessment have 

institutions developed? 

Institutional Support Strategy 

• Most institutions’ mission statements emphasize excellence in undergraduate education and 
many identify intended student outcomes, but few explicitly refer to the importance of student 
assessment. (Ch. 5) 

• Institutions are predominantly engaged in undergraduate student assessment for internal 
purposes of improving undergraduate student achievement, undergraduate academic programs 
and faculty instructional practices. With the clear exception of preparing for accreditation self- 
study, accountability purposes (meeting state reporting requirements and guiding internal 
resource allocation decisions) are comparatively less important. (Ch. 5) 

Leadership and Governance 

• Institutions have instituted a variety of institution-wide administrative and governance 
structures and activities to support student assessment. Institutions are most likely to have 
faculty governance committees and assessment workshops for academic and student affairs 
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administrators, and least likely to offer rewards or incentives for administrators promoting 
assessment in their units. (Ch. 5) 

• The majority of institutions have some form of institutional plan or policy for student 
assessment. This is most often a formal centralized plan or policy in which specified 
assessment activities are required of all academic units or programs. To a lesser extent, 
institutions require all academic units or programs to develop their own undergraduate 
assessment plans or stipulate institution-wide activities to be conducted by a central assessment 
committee or office. (Ch. 5) 

• Governance for student assessment is generally a shared responsibility in institutions, with 
primary responsibility positioned in academic affairs. The majority of institutions have an 
institution-wide student assessment planning group whose representation includes faculty and 
academic affairs administrators and, to a lesser extent, institutional research and student affairs 
personnel. (Ch. 5) 

• Executive responsibility for student assessment planning is most often vested in academic 
affairs administrators, followed by a faculty member. (Ch. 5) 

• Assessment plans and policies are usually subject to approval from multiple sources within the 
institution — particularly the chief academic affairs officer, followed by the chief executive 
officer and academic senate or other faculty committee. (Ch. 5) 

• Operating responsibility for day-to-day student assessment activities is equally likely to be 
given to an academic affairs adm inistrator or institutional research officer and, secondarily, to a 
faculty member. The individual with operational responsibility for student assessment most 
often reports directly to the chief academic officer followed by the chief executive officer. (Ch. 
5) 
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Assessment Management Policies and Practices 



• Almost half of institutions have explicit budget allocations to support student assessment 
activities but very few use student assessment information as a basis for allocating resources to 
academic units. (Ch. 6) 

• Many institutions have information systems capable of tracking students over the course of 
their enrollment but few have computerized student assessment data bases, and even fewer 
have integrated their student assessment data with other institutional data bases. This would 
seem to necessarily limit the capability of institutions to conduct studies of the relationship 
between students’ institutional experiences and performance. (Ch. 6) 

• Institutions are more likely to provide access to assessment information on individual students 
to institutional researchers and academic administrators than to student affairs or faculty 
advisors. They more often distribute assessment reports to academic and student affairs 
administrators and faculty members than to external constituents. These patterns of assessment 
information access and distribution suggest this information is primarily intended to support 
internal administrative or policy-related decisions. (Ch. 6) 

• Institutions have made fairly extensive use of policies that require student involvement in 
assessment activities and provide students with information concerning assessment purposes 
and results. (Ch. 6) 

• Institutions have made less extensive use of professional development policies as a means of 
promoting student assessment. Professional development is primarily offered in the form of 
assessment conferences or workshops. These policies are more often directed toward faculty 
than to academic administrators and student affairs personnel. (Ch. 6) 

• Institutions report encouraging their faculty to assess student learning. However, institutions 
are unlikely to use faculty evaluation and reward policies to promote faculty involvement in 
student assessment. (Ch. 6) 
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• Of all dimensions of assessment management policies and practices considered in this study, 
institutions made the most extensive use of policies incorporating assessment information into 
processes for academic planning and review, particularly with regard to academic programs, 
curriculum and courses. (Ch. 6) 

Evaluation of Student Assessment Process 

• Slightly more than one-quarter of institutions have conducted a formal evaluation of their 
student assessment process while slightly more than one-quarter have conducted an informal 
evaluation. (Ch. 5) 

4. How have institutions used student assessment information and what impacts has it had? 

• Most institutions have either not used assessment information to guide institutional decisions or 
are unaware of the influence of assessment data on institutional decisions. Assessment 
information is most likely to influence decisions concerning the assessment process itself, and 
academic planning at the program, curriculum and classroom levels. Assessment information 
is least likely to influence decisions concerning resource allocations to academic units, and 
faculty evaluation and rewards. (Ch. 7) 

• Most institutions have not monitored the internal institutional impact of student assessment 
information. When they do, institutions most often report that student assessment information 
has led to changes in the teaching methods used by faculty and stimulated campus discussions 
of undergraduate education. Institutions have least often documented positive impacts of 
assessment information on faculty satisfaction and students’ academic performance. (Ch. 7) 

• Institutions are even less likely to monitor the impact of student assessment on external 
indicators of institutional performance. The clear exception to this pattern concerns 
relationships with regional accreditation associations. A large proportion of institutions report 
assessment information has had a positive impact on their evaluation from regional accreditors. 
(Ch. 7) 
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5. How do patterns of external influences, student assessment approach, organizational and 
administrative support, and uses and impacts of student assessment vary by institutional type and 
control? 

Associate of Arts 
External Influences 

• Associate of arts are most likely of all institutional types to report the existence of a state plan or 
requirement for student assessment. Together with research institutions, they are most likely to 
perceive state requirements as a positive influence on their student assessment activities. State 
reporting requirements more often include providing evidence of a student assessment plan and 
use of state-mandated measures than using institutionally-devised performance indicators or 
providing evidence of using assessment information. Of all institutional types, they are most 
likely to have undergone some form of state review of their assessment plans or process. (Ch. 



• Most associate of arts institutions have undergone a regional accreditation review which 
required student assessment. More than state requirements, regional accreditation requirements 
are perceived as having positively influenced institutions’ student assessment efforts. 
Institutions are more often required to provide accreditors with evidence of an assessment plan 
than institutional use of assessment information. (Ch. 4) 

• Few associate of arts institutions have received grants from external sources to improve their 
student assessment practices. (Ch. 4) 

Institutional Approach to Student Assessment 

• Associate of arts institutions are most likely of all types of institutions to collect data on 
students’ entry-level performance and vocational/professional skills and are least likely to 
collect data on students’ cognitive and affective domains. (Ch. 3) 

• If standardized instruments are used, these are more likely to have been developed by the 
institution than purchased commercially or provided by the state. Associate of arts institutions 
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make limited use of nontraditional student assessment methods. They are least likely of all 
institutional types to use student portfolios, capstone courses, or interviews or focus groups 
with current students and alumni. With the exception of distance education students, associate 
of arts institutions rarely use different student assessment methods for special student 
populations. (Ch. 3) 

• With one exception (student exposure to different instructional methods) associate of arts are 
least likely of all institutional types to conduct studies of the relationship between students’ 
institutional experiences and student performance. (Ch. 3) 

• Reports of student assessment results are most often provided at the institution-wide or 
academic program/department level. (Ch. 3) 

• These patterns of student assessment approach are reflective of associate of arts institutions’ 
open-door admissions policies, vocational emphasis and very diverse, mobile student body. 
Organizational and Administrative Support 

• The mission statements of associate of arts institutions typically emphasize excellence in 
undergraduate education and, to a lesser extent, identify intended educational outcomes but less 
often explicitly refer to student assessment as an important institutional activity. (Ch. 5) 

• Associate of arts institutions view student assessment as very important to meeting a number of* 
purposes including reporting to state officials and regional accreditors, improving student and 
faculty performance, and guiding internal resource allocation. (Ch. 5) 

• Associate of arts institutions most often use institution-wide initiatives and assessment 
workshops for administrators as means of promoting student assessment, but rarely use 
incentives for administrators or academic units for this purpose. (Ch. 5) 

• While all internal constituents are reported as being supportive of student assessment, 
administrators are perceived as being most supportive and faculty and students as 
comparatively less supportive. (Ch. 5) 
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• Associate of aits institutions are most likely to use a formal centralized plan or policy for 
student assessment which requires specified assessment activities for all academic units or 
programs. Of all institutional types, they are least likely to use a formal decentralized plan or 
policy in which academic units or programs develop their own student assessment plans. (Ch. 
5) 

• The majority of associate of arts institutions have an institution-wide student assessment 
planning group. Group representation most often includes faculty and academic affairs 
personnel. Associate of arts are more likely than other types of institutions to have the chief 
executive officer, and student affairs and institutional research personnel on this committee. 
Executive responsibility for this group is most often placed with an academic affairs 
administrator. Similarly, approval authority for student assessment plans or policies is most 
often vested in the chief academic affairs officer. Associate of arts institutions are more likely 
than other types of institutions to include the chief executive officer in the approval process and 
less likely to include the academic senate. Oversight of day-to-day assessment activities is 
most often the responsibility of an institutional research officer or academic affairs 
administrator. Overall, governance for student assessment is more often an administrative than 
a faculty responsibility. (Ch. 5) 

• Close to half of associate of arts institutions have an office providing faculty consultation on 
using student assessment. (Ch. 5) 

• About half of associate of arts institutions have formally or informally evaluated their student 
assessment process. Evaluation has most often considered assessment plans or policies, the 
achievement of student assessment objectives, use of assessment information in decision- 
making, and problems encountered in the assessment process. (Ch. 5) 

Assessment Management Policies and Practices 

• Almost half of associate of arts institutions have an explicit budget allocation to support student 
assessment. One-quarter informally consider assessment information when allocating 
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resources to academic units, but very few have formally linked academic resource allocation to 
assessment results. (Ch. 6) 

• Many associate of arts institutions have scheduled student assessment activities into the 
academic calendar. Of all institutional types, these institutions report the most comprehensive 
and sophisticated information systems to support student assessment. (Ch. 6) 

• Compared to other types of institutions, associate of arts institutions provide the greatest 
internal access to individual student assessment information. Student assessment reports are 
more often distributed to internal than external constituencies. (Ch. 6) 

• Associate of arts institutions make moderately extensive use of policies promoting student 
involvement in assessment activities. (Ch. 6) 

• Compared to other types of institutions, associate of arts institutions make the most extensive 
use of professional development policies related to student assessment. Policies are more often 
directed toward faculty than to academic administrators or student affairs personnel. (Ch. 6) 

• Associate of arts institutions encourage faculty to assess student learning. Beyond this, they 
make little use of faculty evaluation and reward policies to promote student assessment. (Ch. 6) 

• Of all types of institutions, associate of arts institutions make the most extensive use of student 
assessment information in academic planning and review processes for academic 
departments/programs, curriculum, courses, and academic support services. (Ch. 6) 
Assessment Uses and Impacts 

• Student assessment data has influenced decisions concerning student assessment plans or 
processes, academic support services, teaching methods, general education curriculum and 
academic programs in some associate of arts institutions. However, the majority of institutions 
either do not use or are unaware of the influence of assessment information on institutional 
decision making. (Ch. 7) 

• Most associate of arts institutions have not monitored the internal impacts of student 
assessment information. Among those institutions where monitoring has occurred, the most 
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frequently documented impacts are changes in teaching methods, and increases in student 
retention or graduation rates, and campus discussions of undergraduate education. (Ch. 7) 

• Many associate of arts institutions report that assessment information has had a positive impact 
on regional accreditation evaluations. With this one exception, most institutions have not 
monitored the impacts of student assessment information on external measures of institutional 
performance. (Ch. 7) 

Baccalaureate Institutions 
External Influences 

• Baccalaureate institutions are least likely of all types of institutions to report the existence of a 
state plan or requirement for student assessment. They are least likely to perceive positive 
influences of state requirements on their assessment efforts. Compared to other institutional 
types, a higher proportion of baccalaureates report they are required to provide evidence of a 
student assessment plan and institutional use of student assessment information to state 
officials. (Ch. 4) 

• The majority of baccalaureate institutions have undergone a regional accreditation review which 
required student assessment. Compared to state assessment requirements, accreditation 
requirements are more often perceived as having positively influenced these institutions’ 
assessment efforts. Institutions are morq often required to provide accreditors with evidence of 
an assessment plan than institutional use of assessment information. (Ch. 4) 

• Among institutional types, baccalaureate institutions are most likely to have received external 
grants to improve their student assessment practices from private or corporate sources, and 
least likely to have received grants from state sources. Still, the majority of baccalaureates have 
not received external grants. (Ch. 4) 

Institutional Approach to Student Assessment 

• Baccalaureate institutions are most likely of all institutional types to collect data on students’ 
cognitive domains (e.g., higher-order cognitive skills, general education, competence in major 
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field), students’ experiences and satisfaction with the institution, and former students’ further 
education and civic/social roles. (Ch. 3) 

• Baccalaureate institutions more often use instruments that have been institutionally or 
commercially developed than state provided. Compared to other institutional types, they make 
greater use of nontraditional student assessment methods such as observations of student 
performance, student portfolios, capstone courses, transcript analysis and surveys/interviews 
with withdrawing students. A small proportion of baccalaureate institutions report using 
different student assessment methods for adult and distance education students. (Ch. 3) 

• Few baccalaureate institutions report conducting studies of the relationship between students’ 
institutional experiences (e.g., exposure to different instructional methods, patterns of student- 
faculty interaction) and student performance. (Ch. 3) 

• Baccalaureate institutions most often provide reports of student assessment results at the 
institution-wide or academic program/department level. (Ch. 3) 

• These patterns of student assessment approach — particularly the types of assessment data 
collected and use of nontraditional assessment methods — are reflective of baccalaureate 
institutions’ emphasis on student development. 

Organizational and Administrative Support 

• Mission statements of baccalaureate institutions usually emphasize excellence in undergraduate 
education and, more often than other institutional types, identify intended educational outcomes 
for students, but seldom make explicit reference to student assessment. (Ch. 5) 

• Baccalaureate institutions identify preparing for regional accreditation the most important 
purpose of student assessment, and give higher importance ratings to improving student 
achievement and faculty instructional performance than other institutional types. They view 
meeting state requirements as a comparatively unimportant purpose of assessment. (Ch. 5) 
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Baccalaureate institutions most often focus on faculty and academic and student affairs 
administrators in their use of administrative and governance activities to promote student 
assessment. (Ch. 5) 

While all internal constituents are reported as being supportive of student assessment, 
administrators are perceived as being most supportive and faculty and students as 
comparatively less supportive. (Ch. 5) 

Baccalaureate institutions are most likely of all institutional types to have a formal centralized 
plan or policy for student assessment. However, they also report using a formal decentralized 
or formal guidance policy approach. (Ch. 5) 

The majority of baccalaureate institutions have an institution-wide planning group for student 
assessment. Group membership most often includes faculty and academic affairs personnel. 
Compared to other types of institutions, baccalaureate institutions least often include student 
affairs or research-oriented personnel in this group. Executive responsibility for this group is 
most often placed with an academic affairs administrator. Approval authority for student 
assessment plans or policies is usually vested in the chief academic affairs officer, followed by 
the academic senate. Day-to-day oversight of assessment activities most often rests with an 
academic affairs administrator or institutional research officer, followed by a faculty member. 
While academic affairs administrators figure prominently in decision making processes, 
governance for student assessment includes a significant degree of faculty participation. (Ch. 
5) 

Baccalaureate institutions are least likely of all institutional types to have an office providing 
faculty consultation on using student assessment. (Ch. 5) 

Slightly more than half of baccalaureate institutions have formally or informally evaluated their 
student assessment process. Compared to other types of institutions, baccalaureates have 
conducted the most comprehensive evaluations, reviewing the largest number of elements of 
the assessment process. (Ch. 5) 



Assessment Management Policies and Prarticec 



Approximately half of baccalaureate institutions have an explicit budget allocation for student 
assessment. They are least likely of all institutional types to informally consider assessment 
information when allocating resources to academic units, and almost none have formally linked 
academic resource allocation to assessment results. (Ch. 6) 

Of all institutional types, baccalaureates are most likely to have scheduled student assessment 
activities into the academic calendar. Comparatively few have student assessment information 
systems that are computerized or integrated with other institutional data bases. (Ch. 6) 

Baccalaureate institutions provide internal constituencies with moderately high access to 
individual student assessment information. Assessment reports are regularly distributed to 
administrators and faculty, but less often to students and least of all to external constituencies. 
(Ch. 6) 

Of all institutional types, baccalaureate institutions make the most extensive use of policies 
promoting student involvement in assessment activities. (Ch. 6) 

Baccalaureate institutions make fairly extensive use of professional development policies 
related to student assessment. Policies are more likely to be directed toward faculty than to 
academic administrators or student affairs personnel. (Ch. 6) 

Compared to other types of institutions, baccalaureate institutions make the most extensive use 
of policies linking student assessment with faculty evaluation and reward policies and 
practices. This most often takes the form of encouraging faculty to assess students, and 
considering faculty assessment-related scholarship and participation in promotion, tenure or 
salary decisions. Even so, use of these policies is generally limited to only a few or some 
departments. (Ch. 6) 

Next to associate of arts institutions, baccalaureate institutions make the most extensive use of 
student assessment information in academic planning and review processes for academic 
departments/programs, curriculum, courses, and academic support services. (Ch. 6) 



Assessment Uses and Impacts 



• Student assessment data has had some influence on decisions to revise academic programs, 
general education curriculum, academic support services, teaching methods, and the academic 
mission. Although baccalaureate institutions reported the most use of assessment information 
among all institutional types, many of these institutions either do not use or are unaware of the 
influence of assessment information on institutional decision making. (Ch. 7) 

• Most baccalaureate institutions have not monitored the internal impacts of student assessment 
information. Among those institutions where monitoring has occurred, the most frequently 
documented impacts are changes in teaching methods, and increases in student retention or 
graduation rates, and campus discussions of undergraduate education. (Ch. 7) 

• Many baccalaureate institutions report that assessment information has had a positive impact on 
regional accreditation evaluations. Save for this exception, most institutions have not 
monitored the impacts of student assessment information on external measures of institutional 
performance. (Ch. 7) 

Master’s Institutions 
External Influences 

• Roughly half of master’s institutions report the existence of a state plan or requirement for 
student assessment. Of these, approximately half perceive state requirements as having 
increased institutional involvement in student assessment. State reporting requirements more 
often include providing evidence of a student assessment plan and use of state-mandated 
measures than use of institutionally-devised performance indicators or institutional use of 
assessment information. (Ch. 4) 

• Most master’s institutions have undergone a regional accreditation review which required 
student assessment. Regional accreditation requirements are more often perceived as a positive 
influence on institutions’ assessment efforts than state requirements. Institutions are more 
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often required to provide accreditors with evidence of an assessment plan than institutional use 
of assessment information. (Ch. 4) 

• The majority of master’s institutions have not received external grants to improve their student 
assessment practices. (Ch. 4) 

Institutional Approach to Student Assessment 

• Master’s institutions are most likely to collect information on current students’ academic 
progress, basic skills and satisfaction with the institution, and least likely to collect information 
concerning students’ vocational/professional skills, affective development or civic/social roles. 
Compared to other types of institutions, these institutions fall in the midrange of use of various 
approaches to student assessment. (Ch. 3) 

• If standardized instruments are used by master’s institutions, these are more likely to be 
institutionally-developed or commercially purchased than provided by the state. Compared to 
other types of institutions, they make moderate use of nontraditional student assessment 
methods. A small proportion of master’s institutions report using different student assessment 
methods for adult and distance education students. (Ch. 3) 

• Master’s institutions report limited attempts to study the relationship between various aspects of 
students’ institutional experiences and students’ performance. (Ch. 3) 

• Reports of student assessment results are most often provided at the level of the whole 
institution or academic programs/departments. (Ch. 3) 

• Overall, master’s institutions tend to fall in the middle range of extensiveness of undergraduate 
student assessment approach. This may be reflective of their need to address both 
undergraduate and graduate education concerns. 

Organizational and Administrative Support 

• Most master’s institutions emphasize excellence in undergraduate education in their institutional 
missions and many identify intended educational outcomes for students, but few explicitly refer 
to student assessment (Ch. 5) 
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Master’s institutions are most likely to view student assessment as meeting academic 
improvement (student and faculty performance) purposes rather than management or external 
accountability purposes. (Ch. 5) 

Like baccalaureate institutions, master’s institutions most often target faculty and academic and 
student affairs administrators in institutional activities promoting student assessment. They are 
most likely of all institutional types to provide student representation on assessment 
committees. (Ch. 5) 

While all internal constituents are reported as being supportive of student assessment, 
administrators are perceived as being most supportive and faculty and students as 
comparatively less supportive. (Ch. 5) 

Master’s institutions are most likely to adopt a formal decentralized plan or policy for student 
assessment in which academic units or programs develop their own assessment plan. Slightly 
fewer report using a formal centralized or formal guidance policy. (Ch. 5) 

Most master’s institutions have an institution- wide planning group for student assessment. 
Group representation most often includes faculty and academic affairs personnel and least often 
includes the chief executive officer. Executive responsibility for this group is most often 
placed with an academic affairs administrator or faculty member. Approval authority for 
student assessment plans or policies most often rests with the chief academic officer followed 
by the academic senate. Oversight of day-to-day assessment activities is generally the 
responsibility of an institutional research officer or academic affairs administrator, although 
faculty members also often hold this position. Overall, master’s institutions share governance 
responsibility for student assessment across administrative and faculty positions. (Ch. 5) 

Approximately half of master’s institutions have an office providing faculty consultation on 
using student assessment. (Ch. 5) 

Approximately half of master’s institutions have evaluated their student assessment process, 
more often informally than formally. Evaluation has most often included a review of 



assessment plans and policies, achievement of intended objectives of assessment, and 
problems encountered in the assessment process. (Ch. 5) 

Assessment Management Policies and Practices 

• Approximately half of master’s institutions have an explicit budget allocation to support student 
assessment. One-quarter informally consider assessment information when allocating 
resources to academic units, but very few have formally linked academic resource allocation to 
assessment results. (Ch. 6) 

• Approximately half of master’s institutions have formally incorporated assessment activities 
into the academic calendar. Most do not have a student assessment information system that is 
computerized or integrated with other institutional data bases. (Ch. 6) 

• Master’s institutions provide internal constituencies with moderately high access to individual 
student assessment information. Assessment reports are regularly distributed to administrators 
and faculty, but less often to students and least of all to external constituencies. (Ch. 6) 

• Master’s institutions make moderately extensive use of policies promoting student involvement 
in assessment activities. (Ch. 6) 

• Master’s institutions make fairly extensive use of professional development policies related to 
student assessment. Policies are more likely to be directed toward faculty or academic 
administrators than to student affairs personnel. (Ch. 6) 

• Master’s institutions encourage their faculty to assess students, and some departments consider 
faculty assessment-related scholarship and participation in promotion, tenure or salary 
decisions. There is scant use of other policies linking student assessment with faculty 
evaluation and rewards. (Ch. 6) 

• Master’s institutions make extensive use of student assessment information in academic 
planning and review processes for academic departments/programs, curriculum, and courses. 
(Ch. 6) 
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Assessment Uses and Impacts 



• Student assessment data has had some influence on decisions to revise academic programs, 
student assessment plans or processes, academic support services, teaching methods, and the 
academic mission in master’s institutions. Still, many of these institutions either do not use or 
are unaware of the influence of assessment information on institutional decision making. (Ch. 



• Most master’s institutions have not monitored the internal impacts of student assessment 
information. Among those institutions where monitoring has occurred, the most frequently 
documented impacts are changes in teaching methods, and increases in student retention or 
graduation rates, and campus discussions of undergraduate education. (Ch. 7) 

• Many master’s institutions report that assessment information has had a positive impact on 
regional accreditation evaluations. Save for this exception, most institutions have not 
monitored the impacts of student assessment information on external measures of institutional 
performance. (Ch. 7) 

Doctoral Institutions 
External Influences 

• Roughly half of doctoral institutions report the existence of a state plan or requirement for 
student assessment Approximately, half Qf these perceive state requirements as having 
increased institutional involvement in student assessment Doctoral institutions are more likely 
to be required to provide state officials with evidence of a student assessment plan than 
institutional use of student assessment information. Of all institutional types, they are most 
likely to report being required to use state-mandated student performance indicators. (Ch. 4) 

• Most doctoral institutions have undergone a regional accreditation review which required 
student assessment. More than state requirements, regional accreditors’ requirements for 
student assessment are more often perceived as having had a positive influence on these 
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institutions’ assessment efforts. Institutions are more often required to provide accreditors 
with evidence of an assessment plan than institutional use of assessment information. (Ch. 4) 

• The majority of doctoral institutions have not received external grants to improve their student 
assessment practices. (Ch. 4) 

Institutional Approach to Student Assessment 

• Doctoral institutions most often collect data on current students’ academic progress, basic 
college-readiness skills and academic intentions and least often collect information on students’ 
higher-order cognitive skills, civic or social roles, and vocational/professional skills. Along 
with master’s institutions, doctoral institutions tend to fall in the midrange of use of various 
approaches to student assessment. (Ch. 3) 

• Doctoral institutions are more likely to use instruments that have been developed by the 
institution itself or commercial sources than state-provided instruments. Compared to other 
types of institutions, they make moderate use of nontraditional student assessment methods. 
With the exception of distance education students, doctoral institutions do not use different 
student assessment methods for special student populations. (Ch. 3) 

• Compared to other types of institutions, a moderate proportion of doctoral institutions conduct 
studies of the relationship between various aspects of students’ institutional experiences and 
student performance. (Ch. 3) 

• Doctoral institutions most often provide reports of student assessment results at the institution- 
wide, academic program/department, and school/college levels. (Ch. 3) 

• Doctoral institutions tend to fall in the midrange of extensiveness of undergraduate student 
assessment approach. This may be attributable to their dual focus on undergraduate and 
graduate education. 
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Organizational and Administrative Supp ort 

Most doctoral institutions emphasize excellence in undergraduate education in their institutional 
missions and, to a lesser extent, identify intended educational outcomes for students but fewer 
explicitly refer to student assessment. (Ch. 5) 

Doctoral institutions view preparing for regional accreditation, improving student achievement 
and academic programs as important purposes of student assessment but give meeting state 
requirements, improving faculty instruction and guiding internal resource allocation lower 
importance ratings. (Ch. 5) 

Doctoral institutions most often focus administrative and governance activities promoting 
student assessment toward faculty and academic and student affairs administrators. They are 
more likely than other types of institutions to use incentives to encourage administrators and 
academic units to use assessment information. (Ch. 5) 

Like other types of institutions, doctoral institutions report all internal constituents as being 
supportive of student assessment, but administrators are perceived as being most supportive 
and faculty and students as comparatively less supportive. (Ch. 5) 

Like master’s institutions, doctoral institutions are most likely to have a formal decentralized 
student assessment policy in which academic units or programs determine their own 
assessment plans. A somewhat smaller proportion use a formal guidance or formal centralized 
plan. (Ch. 5) 

A large proportion of doctoral institutions have an institution-wide planning group for student 
assessment. This group is most often comprised of faculty members and academic affairs 
personnel. Compared to other types of institutions, doctoral institutions are least likely to have 
the chief executive officer as a group member and most likely to include student 
representatives. Executive responsibility for this group is equally likely to be given to an 
academic affairs administrator or faculty member. Approval authority for student assessment 
plans or policies is most often vested in the chief academic affairs officer and less often 



includes the academic senate. Day-to-day oversight of assessment activities is primarily the 
responsibility of an academic affairs administrator, followed by institudonal research officers 
and faculty members. Overall, doctoral institutions employ a broad range of internal 
representation in student assessment governance. (Ch. 5) 

• Half of doctoral institutions have an office providing faculty consultation on using student 
assessment. (Ch. 5) 

• Approximately half of doctoral institutions have evaluated their student assessment process, 
more often informally than formally. Evaluation has most often reviewed assessment plans 
and policies, problems encountered in the assessment process, assessment structure and 
governance, and achievement of intended objectives of assessment. (Ch. 5) 

Assessment Management Policies and Practices 

• Almost half of doctoral institutions have an explicit budget allocation for student assessment. 
Some informally consider assessment information when allocating resources to academic units, 
but very few have formally linked academic resource allocation to assessment results. (Ch. 6) 

• Approximately half of doctoral institutions have scheduled student assessment activities into the 
academic calendar. Most do not have a student assessment information system that is 
computerized or integrated with other institutional data bases. (Ch. 6) 

• Compared to other types of institutions, doctoral institutions provide internal constituencies 
with slightly less access to individual student assessment information. Assessment reports are 
regularly distributed to administrators and faculty, but less often to students and least of all to 
external constituencies. (Ch. 6) 

• Doctoral institutions make fairly extensive use of policies promoting student involvement in 
assessment activities. (Ch. 6) 

• Doctoral institutions make moderate use of professional development policies related to student 
assessment. Policies are more likely to encourage rather than require participation in 
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professional development, and are more often directed toward faculty than to academic 
administrators or student affairs personnel. (Ch. 6) 

• Doctoral institutions encourage their faculty to assess students, and give some consideration to 
assessment-related scholarship in faculty promotion, tenure or salary decisions. They make 
scant use of other policies linking student assessment with faculty evaluation and rewards. 
(Ch. 6) 

• Doctoral institutions make fairly extensive use of student assessment information in academic 
planning and review processes for academic departments/programs, curriculum, and courses. 
(Ch. 6) 

Assessment Uses and Impacts 

• Student assessment data has had some influence on decisions to revise student assessment 
plans or processes, academic support services, teaching methods, academic programs, and the 
general education curriculum in doctoral institutions. But the majority of these institutions 
either do not use or are unaware of the influence of assessment information on institutional 
decision making. (Ch. 7) 

• The majority of doctoral institutions have not monitored the internal impacts of student 
assessment information. Among those institutions where monitoring has occurred, the most 
frequently documented impacts are changes in teaching methods, and increases in student 
retention or graduation rates and campus discussions of undergraduate education. (Ch. 7) 

• Many doctoral institutions report that assessment information has had a positive impact on 
regional accreditation evaluations. Save for this exception, most institutions have not 
monitored the impacts of student assessment information on external measures of institutional 
performance. (Ch. 7) 
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Research Institutions 



External Influences 

• Roughly half of research institutions report the existence of a state plan or requirements for 
student assessment. Together with associate of arts institutions, they are most likely to 
perceive state assessment requirements as a positive influence on their student assessment 
activities. Like other types of institutions, research institutions are more likely to be required to 
provide state officials with evidence of a student assessment plan than of institutional use of 
assessment information. They are most likely to be required to report on institutionally-devised 
student performance indicators. (Ch. 4) 

• The majority of research institutions have undergone a regional accreditation review which 
required student assessment. Compared to other types of institutions, these institutions appear 
to be less affected by accreditation requirements for student assessment They are significantly 
less likely to perceive positive influences on their assessment activities attributable to regional 
accreditation requirements and perceive fewer reporting requirements stemming from regional 
accreditors. (Ch. 4) 

• Of all institutional types, research institutions are most likely to have received a grant from 
some external source to improve their student assessment practices. Still, almost three-quarters 
of research institutions have received no external grants for this purpose. (Ch. 4) 

Institutional Approach to Student Assessment 

• Research institutions report a mix of data collection efforts compared to other types of 
institutions, scoring lowest on six and highest on three of the fourteen assessment measures 
considered in our study. They are most likely to collect data on current students’ academic 
progress, academic intentions and basic college-readiness skills, and least likely to collect data 
on students’ vocational/professional skills, civic/social roles and higher-order cognitive skills. 
(Ch. 3) 
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Research institutions are more likely to use institutionally-developed instruments than those 
provided by the state or commercial sources. Like associate of arts institutions, they make 
limited use of nontraditional student assessment methods. Compared to other types of 
institutions, they are least likely to use observations of student performance, transcript 
analysis, and external examinations of students. With the exception of distance education 
students, research institutions make limited use of different student assessment methods for 
special student populations. (Ch. 3) 

Overall, research institutions are most likely of all institutional types to conduct studies of the 
relationship between students’ institutional experiences and student performance. This may 
reflect the capacity of institutional research offices at these institutions. (Ch. 3) 

Research institutions are most likely to provide reports of student assessment results at the 
levels of schools/colleges or the institution as a whole. (Ch. 3) 

With the exception of conducting student assessment studies, research institutions generally 
report less extensive undergraduate student assessment approaches than other types of 
institutions. These patterns of student assessment approach are likely attributable to their 
emphasis on graduate as opposed to undergraduate education. (Ch. 3) 

Oreanizational and Administrative Support 

The institutional missions of rrtost research institutions emphasize excellence in undergraduate 
education, but compared to other institutional types, they are least likely to identify intended 
educational outcomes for students or explicitly refer to student assessment. (Ch. 5) 

Research institutions view improving academic programs and student achievement as very 
important purposes of their student assessment efforts but generally ascribe less importance to 
student assessment purposes than other types of institutions. (Ch. 5) 

Compared to other types of institutions, research institutions make the highest use of 
assessment workshops and rewards or incentives for administrators as a means of promoting 
student assessment. (Ch. 5) 



Compared to other types of institutions, research institutions report somewhat lower levels of 
support for student assessment among various groups of internal constituents. Similar to other 
institutions, administrators are perceived as being most supportive and faculty and students as 
comparatively less supportive of student assessment. (Ch. 5) 

Research institutions are more likely than other institutional types to have an informal or no 
undergraduate student assessment plan or policy in place. Those institutions with plans are 
most likely to decentralize decisions concerning student assessment to academic units or 
programs. (Ch. 5) 

Less than half of research institutions have an institution-wide planning group for student 
assessment, the smallest proportion of all institutional types. These institutions are most likely 
of all types to include personnel with specialized assessment or institutional research expertise 
in a planning group and least likely to have faculty representation. Executive responsibility for 
this group is most often vested in an academic affairs administrator and, compared to other 
institutional types, least often placed with a faculty member. Approval authority for student 
assessment plans or policies usually rests with the chief academic affairs officer and, compared 
to other institutional-types, less often includes the academic senate. Oversight for day-to-day 
assessment activities is usually the responsibility of an institutional research officer or academic 
affairs administrator; compared to other institutional types, research institutions are least likely 
to assign this responsibility to a faculty member. Overall, governance patterns for student 
assessment primarily emphasize administrative and, to a lesser extent, specialized research 
involvement while faculty members have comparatively less involvement (Ch. 5) 

Almost two-thirds of research institutions have an office providing faculty consultation on 
using student assessment, the highest proportion among all types of institutions. (Ch. 5) 

Of all institutional types, research institutions are least likely to have formally or informally 
evaluated their assessment process, and most likely not to be planning such an evaluation. 
When conducted, evaluations most often review assessment plans and policies, the use of 



assessment information in decision-making, and problems encountered in the assessment 
process. (Ch. 5) 

Assessment Management Policies and Practices 

Only one-third of research institutions have an explicit budget allocation for student 
assessment, the smallest proportion of any institutional type. Some informally consider 
assessment information when allocating resources to academic units, but very few have 
formally linked academic resource allocation to assessment results. (Ch. 6) 

Of all institutional types, research institutions are least likely to have scheduled student 
assessment activities into the academic calendar. A moderately high proportion have 
computerized student assessment information systems. (Ch. 6) 

Compared to other types of institutions, research institutions provide internal constituencies 
slightly less access to individual student assessment information. Assessment reports are 
regularly distributed to administrators, less often to faculty or students, and least of all to 
external constituencies. (Ch. 6) 

Of all institutional types, research institutions make the least extensive use of policies 
promoting student involvement in assessment activities. (Ch. 6) f 

Compared to other types of institutions, research institutions make the least extensive use of 
professional development policies related to student assessment. Policies are more likely to 
encourage rather than require participation in professional development. (Ch. 6) 

Research institutions encourage their faculty to assess student learning. Of all institutional 
types, they make the least extensive use of policies linking student assessment with faculty 
evaluation and rewards. (Ch. 6) 

Research institutions make moderately extensive use of student assessment information in 
academic planning and review processes for academic departments/programs and core 
curriculum. Of all types of institutions, they are least likely to incorporate assessment 
information into academic planning and review processes. (Ch. 6) 



Assessment Uses and Impacts 



• Student assessment data has had some influence on decisions to revise academic support 
services, academic programs, teaching methods, student assessment plans or processes, and 
general education curriculum in research institutions. But even more than other institutional 
types, research institutions either do not use or are unaware of the influence of assessment 
information on institutional decision making. (Ch. 7) 

• Of all institutional types, research institutions are least likely to monitor the internal impacts of 
student assessment information. Among those institutions where monitoring has occurred, the 
most frequently documented impacts are changes in teaching methods, and increases in student 
retention or graduation rates and campus discussions of undergraduate education. (Ch. 7) 

• Research institutions report that assessment information has had a positive impact on regional 
accreditation evaluations. Save for this exception, most institutions have not monitored the 
impacts of student assessment information on external measures of institutional performance. 
(Ch. 7) 

Institutional Control (Public and Private’) 

External Influences 

• The existence and influence of state assessment plans is largely restricted to public institutions. 
This accounts for the differences in perceived external influences reported by baccalaureate 
(largely private) and associate of arts (largely public) institutions. (Ch. 4) 

• Although the majority of institutions associate regional accreditation requirements with positive 
influences on their assessment activities, private institutions are more likely to do so than public 
institutions. (Ch. 4) 

• Public institutions are more likely than private to have received external grants to improve their 
student assessment practices from state programs and federal agencies other than FIPSE, while 
private institutions are more likely than public to have received such grants from private 
foundations or corporate sources. (Ch. 4) 



Institutional Approach to Student Assessment 

• Compared to public institutions, private institutions collect more extensive student assessment 
data, make more extensive use of nontraditional student assessment methods, are more likely to 
use different assessment methods for special student populations, and more often conduct 
studies of the relationships of students’ institutional experiences to student performance. (Ch. 



• Public institutions are more likely than private to provide reports of student assessment 
information at the institution-wide and course level, and for special student subpopulations. 
(Ch. 3) This difference may be due to the greater pressure for accountability from state 
officials experienced by public institutions. 

Organizational and Administrative Support 

• Compared to public institutions, private institutions are more likely to emphasize excellence in 
undergraduate education and identify intended educational outcomes for students, but public 
institutions are more likely to explicitly refer to student assessment as an important institutional 
activity. (Ch. 5) 

• Private institutions emphasize internal improvement purposes of student assessment more than 
public institutions. Public institutions are more likely than private to view meeting state 
requirements.as an important purpose of assessment. (Ch.. 5) 

• Private institutions report greater faculty support for student assessment than public 
institutions. (Ch. 5) 

• Private institutions are more likely than public to have a formal decentralized plan or policy for 
student assessment and to be currently developing their student assessment plan or policy. (Ch. 



• Public and private institutions are equally likely to have an institution-wide planning group for 
student assessment. Public institutions are more likely than private to have student affairs, 
institutional research and student assessment personnel represented on this group, and to place 
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executive responsibility for this group with a student affairs administrator. Public institutions 
are more likely than private to vest approval authority for assessment plans or policies in 
administrative positions other than the chief academic officer and less likely to involve the 
academic senate. Responsibility for planning and directing ongoing assessment activities is 
more often vested in research expertise positions in public institutions while private institutions 
are more likely to give this responsibility to an academic administrator or faculty member. (Ch. 
5 ) 

Public institutions are more likely than private to have an office providing faculty consultation 
on using student assessment. (Ch. 5) 

Public institutions are slightly more likely than private to have evaluated their student 
assessment process while private institutions are slightly more likely to be developing plans for 
such an evaluation. (Ch. 5) 

Assessment Management Policies and Practices 

Public institutions are more likely, than private institutions to informally consider student 
assessment results in determining resource allocations for academic units. (Ch. 6) 

Private institutions are more likely than public to have scheduled student assessment activities 
into the academic calendars. Public institutions are more likely than private to have a 
computerized. information system for student assessment. (Ch. 6) 

Public institutions are more likely than private to provide institutional researchers and student 
affairs professionals with access to assessment information on individual students, and to 
distribute student assessment reports to student affairs professionals, employers and the 
general public. (Ch. 6) 

Compared to public institutions, private institutions make slightly more extensive use of 
policies requiring students to participate in assessment activities and providing incentives to 
encourage student participation. (Ch. 6) 



• Public institutions are more likely than private to have professional development policies related 
to student assessment. (Ch. 6) 

• Private institutions make slightly more extensive use of policies linking student assessment 
with faculty evaluation and rewards than public institutions. (Ch. 6) 

• Public and private institutions do not differ significantly in their incorporation of student 
assessment information into processes for academic planning and review. (Ch. 6) 

Assessment Uses and Impacts 

• There are few differences between public and private institutions in their use of assessment 
information in institutional decision making, and monitoring of impacts of assessment 
information. (Ch. 7) 

6. How are external influences related to the institutional adoption of various approaches to 

student assessment, patterns of organizational and administrative support for assessment, and uses 

and impacts of student assessment information? 

Relationship to Institutional Approach to Assessment 
State Influences 

• There are significant differences in the form of student assessment initiative and standardization 
of reporting requirements associated with the authority of the state governance structure for 
higher education. (Ch. 9) 

• Differences in state approaches to assessment are associated with differences in institutions’ 
approaches to assessment. Of three dimensions of state assessment approaches considered — 
governance structure, form of assessment initiative, and use of common indicators/outcomes 
— governance structure for higher education is most often associated with differences in 
institutions’ assessment approaches. (Ch. 9) 

• Institutions in states using coordinating regulatory boards collect the most extensive assessment 
data while those in states with coordinating advisory boards collect the least. There are fewer 
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differences in institutions’ patterns of data collection by form of state assessment initiative and 
state reporting requirements for assessment. (Ch. 9) 

• Institutions in states with coordinating advisory boards for higher education use a smaller 
number of student assessment data collection methods while institutions in states with planning 
agencies use the greatest variety of data collection methods. Institutions in states with 
assessment statutes are less likely to use student-centered assessment methods (e.g., 
portfolios, capstone courses). (Ch. 9) 

• There is little relationship between state assessment approaches and the assessment studies and 
reports produced by institutions. This aspect of institutions’ student assessment approach is 
largely unrelated to state influences. (Ch. 9) 

Accreditation Influences 

• There are statistically significant differences in institutions’ approaches to student assessment 
by accreditation region. Institutions in the Middle States, Southern and North Central 
accrediting regions have the most extensive data collection efforts. Institutions in the Southern 
and North Central regions make greatest use of a variety of data collection methods. 

Institutions in the Western accrediting region collect the least extensive assessment data and 
make the least extensive use of various data collection methods. (Ch. 9) 

Relationship to Organizational and Administrative Support for Assessment 
State Influences 

• Variations in the extent of organizational and administrative support for assessment are more 
often associated with differences in state governance structures than with differences in the 
form of state assessment initiatives or state reporting requirements for assessment. (Ch. 9) 

• There is a positive association between the authority of the state governance structure for higher 
education and the strength or extent of institutions’ assessment support strategies. Institutions 
in states with coordinating regulatory and consolidated governing boards generally score higher 
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on support strategy dimensions than institutions from states with planning agencies and 
coordinating advisory boards for higher education. (Ch. 9) 

Institutions are most likely to report that meeting state requirements is an important purpose of 
their student assessment efforts if they are in states with coordinating regulatory or 
consolidated governing boards, the state assessment initiative is in the form of a statute, and 
institutions are required to report institutionally-devised indicators or outcomes. (Ch. 9) 

The authority of the state governance structure for higher education is positively associated 
with administrative and faculty support for assessment. (Ch. 9) 

The authority of the state governance structure for higher education is positively associated 
with institutions’ use of formal centralized plans or policies for student assessment. (Ch. 9) 

In general, institutions in states with planning agencies for higher education make the most 
extensive use of assessment management policies and practices to support their student 
assessment efforts while those in states with coordinating advisoiy boards make the least 
extensive use. (Ch. 9) 

Institutions permitted to report institutionally-specific student indicators and outcomes make 
greater use of professional development policies than institutions required to report common 
indicators/outcomes. (Ch. 9) 

Accreditation Influences 

In general, institutions in the Southern accrediting region report the highest scores on 
institutional support strategy dimensions while institutions in the Western accrediting region 
report the lowest. (Ch. 9) 

Institutions in the Southern accrediting region report the most extensive use of assessment 
management policies and practices. Patterns of using assessment management policies and 
practices vary considerably among institutions in the other accrediting regions, but institutions 
in the Western region have the lowest use scores overall. (Ch. 9) 



R elationship to Institutional Uses and Impacts of Assessment 
State Influences 

• There is little association between state assessment approaches and institutional uses and 
impacts of assessment information. There are positive relationships between institutions’ 
documentation of positive external impacts from assessment and two dimensions of state 
approach — the authority of the state governance structure for higher education, and the use of 
statutes. (Ch. 9) 

Accreditation Influences 

There are small but significant associations between regional accrediting affiliation and 
institutional uses and impacts of assessment information. Institutions in the Southern region 
report the greatest influence of assessment information on academic and faculty decisions, and 
have documented the most positive student and external impacts. Compared to institutions in 
other accrediting regions, those in the New England region report the lowest use and impacts 
from assessment. (Ch. 9) 

7. How are institutional approaches to and organizational and administrative support patterns 
for student assessment related to uses and impacts of student assessment information? 

• The extensiveness of institutions’ data collection efforts, particularly the extent to which they 
collect data on students’ cognitive domains, is strongly associated with student assessment 
uses and impacts. (Ch. 10) 

• Conducting assessment for internal purposes and the degree of administrative and faculty 
support for assessment are strongly associated with using assessment information for academic 
decisions. (Ch. 10) 

• Providing professional development on assessment for academic administrators, faculty, and 
student affairs personnel, and incorporating assessment information into processes for 
academic planning and review are strongly associated with using assessment information for 
academic decisions. (Ch. 10) 



• Linking student assessment with faculty evaluation criteria is strongly associated with using 
assessment information for faculty decisions. (Ch. 10) 

• Incorporating assessment information into processes for academic planning and review is 
strongly associated with achieving faculty, student and external impacts from assessment 
information. (Ch. 10) 

8. What is the relative influence of external groups, institutional approach to assessment, and 

patterns of organizational and administrative support for assessment on institutional uses and 

impacts of student assessment information? How does this influence vary by institutional type? 
Overall Influence 

• Overall, the domains of assessment management policies and practices, institutional support 
strategy for assessment, and student assessment approach are stronger predictors of 
assessment uses and impacts than the domains of institutional characteristics and external 
influences. This suggests that institutions can enhance the likelihood of using and achieving 
positive impacts from their student assessment efforts irrespective of their broad characteristics 
or external context. (Ch. 11) 

• Two specific institutional practices and policies appear as the most important determinants of 
assessment uses and impacts: incorporating assessment information into academic planning 
and review processes and the number of assessment studies conducted. These indexes reflect 
the extent to which institutions build formal linkages between their student assessment efforts 
and process for institutional decision-making, and the extent to which institutions analyze the 
relationships between various aspects of students’ institutional experiences and their 
performance. (Ch. 11) 

Influence on Academic Decisions 

• Institutions are more likely to use assessment information in academic decisions (academic 
planning at the department, program, curriculum and course levels; assessment plans and 
processes; resource allocation) if they conduct assessment for internal improvement purposes. 
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provide professional development on assessment for student affairs personnel, and conduct 
assessment studies. (Ch. 11) 

Influence on Faculty Decisions 



• Institutions are more likely to use assessment information in faculty evaluation and reward 
decisions if they incorporate assessment information into academic planning and review 
processes, use student-centered assessment methods, and conduct assessment studies. (Ch. 

11 ) 

Influence on Faculty Impacts 

• Institutions are more likely to achieve positive faculty impacts (satisfaction, discussions of 
undergraduate education, interest in teaching, changes in teaching methods) from assessment if 
they conduct assessment studies, incorporate student assessment information into academic 
planning and review processes, sponsor administrative and governance activities to promote 
student assessment, and provide professional development on assessment for academic 
administrators and faculty. (Ch. 1 1) 

Influence on Student Impacts 

• Institutions are more likely to achieve positive student impacts (retention or graduation rates, 
grade performance, achievement on external examinations, satisfaction) from assessment if 
they incorporate assessment information into academic planning and review processes, conduct 
assessment studies, and provide professional development on assessment for student affairs 
personnel. (Ch. 11) 

Influence on External Impacts 

• Institutions are more likely to achieve positive external impacts (relationships with private and 
public funding sources, regional accreditation evaluations, prospective students, community) 
from assessment if they incorporate assessment information into academic planning and review 
processes, conduct assessment studies, provide professional development on assessment for 
academic administrators and faculty, and distribute assessment reports. 
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12.4 Conclusions 



While the overall picture is one of an evolutionary pattern of student assessment — positive 
but limited influences from state officials and regional accrediting bodies, widespread institutional 
engagement in student assessment but generally emphasizing a constricted array of measures or 
methods, moderate organizational and administrative support to promote student assessment, and 
little use of student assessment data or monitoring of assessment impacts — there are practical and 
scholarly implications of this survey. 

First, the conceptual framework for examining an institution’s engagement with student 
assessment appears to be useful (see Figure 1.2). Institutions reported student assessment-related 
activities in all five domains. The framework and the variables it includes provide a comprehensive 
means for examining an institutional profile of approach to and organizational and administrative 
support for student assessment. This should be beneficial to scholars examining how institutions 
promote and use student assessment. 

Second, the survey instrument (ISSA) provides a useful inventory or checklist for 
individual institutions to examine the assessment approach (measures and data collection methods) 
they are using and the patterns of organizational and administrative support activities in which they 
are engaged. Specifically, it allows them to highlight areas in which they might add to their 
assessment approach, introduce new institutional support activities or assessment management 
policies and practices, and identify areas in which to monitor assessment uses and impacts. 

Third, the survey provides a baseline for monitoring student assessment activity nationally. 
Periodic follow-up surveys would be useful and should provide an opportunity to examine the 
direction and nature of institutions’ future student assessment efforts. 

Fourth, the extensive differences by institutional type on their approaches to, support for, 
and uses and impacts of student assessment suggest the need for more intensive examination of 
student assessment issues by institutional type. Clearly, institutions have differing educational 
missions and clientele which influence their approach to student assessment, how they can best 
support and promote assessment efforts, and how the information can be used most effectively. 
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Fifth, the limited relationship between the influences of state assessment policy and 
accreditation region on institutional approaches to, support patterns for, and, particularly, uses and 
impacts of student assessment data was surprising. Plainly, there is a need for more extensive and 
focused study of the linkage between state-level policy and institutional actions, and of the impacts 
of regional accreditors’ student assessment requirements on institutions. These external 
constituents have devoted extensive efforts to addressing student assessment. Greater 
understanding of how to make their efforts more effective in improving institutional involvement 
with, support for, and uses of student assessment is needed. 

Sixth, the relatively low level of institutional use of student assessment information in 
institutional decision-making and the very limited attempts by institutions to monitor assessment 
impacts suggests the need for greater efforts in this domain of organizational and administrative 
support for assessment. Institutions do not routinely use student assessment data in internal 
decision-making or monitor its impact on important areas of institutional and student performance. 
Given the extensive claims made for the value of student assessment and the substantial human and 
financial resources invested in student assessment activities, institutions need to give greater 
priority to examining how student assessment data is used, and how it impacts the performance of 
individual students and the institution itself. 

Finally, the relatively low level of variance accounted for in the models which examined the 
relationship of various dimensions of external influences, assessment approach, and organizational 
and administrative support for assessment to measures of assessment uses and impacts can be 
partially accounted for by the low degree of institutional use and monitoring of impacts (i.e., 
limited variance of the dependent measures). However, it also implies there are other dimensions 
and dynamics that influence the extent to which institutions use and achieve positive impacts from 
student assessment information. In particular, the culture and climate for student assessment (a 
domain identified in the literature review and conceptual framework but not included in this study), 
the dynamics of academic units, instructional patterns, and the role of faculty in the adoption and 
use of student assessment need to be examined more closely. These dimensions will be examined 
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more closely in the next phase of our research, intensive comparative case studies of institutional 
processes related to student assessment. 

The results of this research suggest that student assessment is becoming a common practice 
in the academic management of postsecondary education in the United States. However, it is not 
yet institutionalized or deeply embedded in institutional support patterns, policies and practices. 
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Inventory of 
Institutional Support for 
Student Assessment 



For The Research Program on 
Institutional Support for Student Assessment 




NCPI 

1 National 
Center for 
Postsecondary 
Improvement 



NCPI - Project 5.2 
University of Michigan 
Ann Arbor, Michigan 481 09- 1259 




An Introduction to the ISSA 



The Institutional Support for Student Assessment Inventory (ISSA) was developed as part of a national research 
program examining the Organizational and Administrative Support for Student Assessment for the National Center 
for Postsecondary Improvement (NCPI). The ISSA is designed as an institutional inventory of the organizational 
and administrative practices that have been designed and implemented to support the use of student assessment on 
your campus. 

Institutional Support Practices are those organized activities, policies, and procedures that your institution has 
intentionally designed to enhance the practice of student assessment Student Assessment refers to those activities 
focused on measuring dimensions of student performance other than traditional end of course grading. 

This national survey is designed to identify institutional support practices for undergraduate student assessment. The 
project also examines the factors influencing the adoption of various support practices and how those practices 
enhance the impact of student assessment for institutional improvement. 

We understand that being selected for this survey will require a commitment of time to complete and we appreciate 
your involvement. This instrument is also intended as an institutional self-assessment inventory to facilitate 
examination of your institution’s own organizational and administrative practices which support student assessment 
We encourage each institution to use the survey in this manner. You will receive a summary report of survey 
responses to all compare with your own institutional profile. 

Completing the ISSA 

The main purpose is to obtain a profile of your institution’s current approach to undergraduate student assessment 
and its support practices. The inventory may be completed by one individual or group of individuals who are most 
familiar with the patterns of undergraduate student assessment on your campus. It should take less than one hour to 
complete. 

• Please keep in mind that the questions refer to undergraduate education at your institution. 

• Respond to each item in the questionnaire to the best of your knowledge. 

The questionnaire is coded to allow follow up only. Individual institutions will not be identified in any analyses or 
reports. 

Return the completed questionnaire in the enclosed return envelope. Any questions concerning the survey can be 
addressed to the following: 



National Center for Postsecondary Improvement Project 5.2 
School of Education 
University of Michigan 
610 E. University, Room 2339 
Ann Arbor, MI 48109-1259 
Phone: 734-647-2464 
Fax: 734-936-2741 
Email: ncpi.proj52@umich.edu 



Marvin W. Peterson, Project Director 



NCPI is funded by the U.S. Department of Education’s Office of Educational Research and Improvement under OERI 

grant number R309A60001 

©1997, The Regents of the University of Michigan 
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L Institutional Approach to Student Assessment 



A. Type, Extent and Timing of Student Assessment 



We are interested in your institution’s routine practices of collecting different types of undergraduate student 
performance data, the extent to which they are collected, and when they are collected. For each of the following 
content types of undergraduate student performance data: 

1) indicate the extent to which each type is collected 

2) for each type of data collected, check whether it is collected at entry, during enrollment, at exit, or a 
combination of these data collection points. 



Type 



Currently Enrolled Students 



1. Student academic intentions or 
expectations 

2. Basic college-readiness 
skills (reading, writing, 
mathematics, etc.) 

3. Higher-order skills (critical 
thinking, problem solving) 

4. General education 
competencies 

5. Competence in major field 
of study (discipline- or 
program-specific knowledge) 

6. Vocational or professional 
skills 

7. Personal growth and 
affective development 
(values, attitudes, social 
development, etc.) 

8. Student experiences and 
involvement with institution 

9. Student satisfaction with 
institution 

10. Student academic progress 
(retention, graduation rates) 

Former Students 

1 1. Vocational or professional 
outcomes (career goals, job 
attainment or performance) 

12. Further education 
(transfer, degree attainment, 
graduate study) 

13. Civic or social roles 
(political, social or 
community involvement) 

14. Satisfaction and experiences 
with institution after leaving 





Extent 






Timing 




Collected 
Not for some 

Collected students 


Collected 
for many 
students 


Collected 
for all 
students 


Collected 

at 

entry 


Collected 

while 

enrolled 


Collected 

at 

exit 




(circle number for each item) 


(check all that apply for each item) 


1 


2 


3 


4 


— 


— 


— 


1 


2 


3 


4 


— 


— 


— 


1 


2 


3 


4 


— 


— 


— 


1 


2 


3 


4 


— 


— 


— 


1 


2 


3 


4 


— 


— 


— 


1 


2 


3 


4 


— 


— 


— 


1 


2 


3 


4 


— 


— 


— 


1 


2 


3 


4 


___ 








1 


2 


3 


4 


— 


— 


— 


1 


2 


3 


4 








1 


2 


3 


4 








1 


2 


3 


4 








1 


2 


3 


4 








1 


2 


3 


4 
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B. 



Student Assessment Instruments 



Does your institution employ institutionally or externally developed instruments or tests for the following types of 
undergraduate student assessment information? (circle gU that apply for each item): 

Source of Instrument 



Content of Instrument 


Not 

used 


Institutionally 

developed 


State 

provided 


Commercially 

available 


1. Student plans, goals, or expectations 


1 


2 


3 


4 


2. Basic college-readiness skills 

(reading, writing, mathematics, etc.) 


1 


2 


3 


4 


3. Higher-order skills 

(critical thinking, problem solving) 


1 


2 


3 


4 


4. General education competencies 


1 


2 


3 


4 


5. Competence in major field of study 
(discipline- or program-specific knowledge) 


1 


2 


3 


4 


6. Vocational or professional skills (excluding 
licensure exams) 


1 


2 


3 


4 


7. Personal growth and affective development 
(values, attitudes, social development, etc.) 


1 


2 


3 


4 


8. Student effort, experiences or involvement 
with institution 


1 


2 


3 


4 


9. Student satisfaction with institution 


1 


2 


3 


4 


10. Alumni satisfaction and experiences 
C. Other Student Assessment Methods 


1 


2 


3 


4 



To what extent does your institution use the following methods to collect undergraduate student assessment 
information? (circle fine number for each item): 



Other Student Assessment Methods 



Not Used in Used in Used in 

used some units 0 most units all units 



1. Observations of student performance 12 3 4 

(simulations, demonstrations, lab) 

2. Student portfolios or comprehensive projects 12 3 4 



3. Student performance in capstone courses 



2 3 4 



4. Student interviews or focus groups 



2 3 4 



5. Transcript analysis 



2 3 4 



6. External examination of students 
(licensure exams, external reviewers) 

7. Special surveys of or interviews with 
withdrawing students 

8. Alumni interviews or focus groups 



12 3 4 

12 3 4 



1 2 3 



4 



9. Employer interviews or focus groups 



2 3 4 



* “Unit” refers to academic areas such as departments, divisions, schools, or colleges. 



D. 



Student Sub-Populations 



Does your institution use different assessment methods for the following sub-populations of undergraduate students? 
(check fine for each item): 

Different Same as Other 

Students 

1. Adult students 

2. Part-time students 

3. Minority students 

4. Distance education students 

E. Student Assessment Studies 

Does your institution conduct studies of the relationship between the following experiences and students’ 
performance (check all that apply): 

1. Student course- taking patterns 

2. Exposure to different instructional or teaching methods 

3. Patterns of student-faculty interaction 

4. Extra-curricular activities 

5. Residence arrangements 

6. Student financial aid and/or concurrent employment 

7. Admission standards or policies 

8. Academic advising patterns 

9. Classroom, library and/or computing resources 

10. Do not study the relationship between the above experiences and student performance 

F. Student Performance Profiles or Reports 

Does your institution provide profiles or reports of appropriate student performance information at the following 
levels of aggregation (check ah that apply): 

1. Institution wide 

2. Schools or colleges 

3. Academic programs or departments 

4. Special populations or subgroups/students ^ • - 

5. By course or groups of courses 

6. Do not provide any reports 



II. Institutional Support for Student Assessment 
A . Institutional Emphasis 

1 . Your institutional mission statement explicitly (check Ml that apply): 

a. emphasizes excellence in undergraduate education as an institutional priority 

b . identifies the educational outcomes intended for your students 

c. refers to student assessment as an important institutional activity 

d. does not explicitly mention any of the above 

2. For how many years has your institution engaged in student assessment? 
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B. 



Purpose of Student Assessment 



The following are often intended purposes of an institution’s undergraduate student assessment process. Please rate 
tne importance of each for your institution, (circle fine number for each item): 



Purpose 


No 

Importance 


Minor 

Importance 


Moderate 

Importance 


Very 

Important 


1. Preparing institutional self-study 


1 


2 


3 


4 


for accreditation 
2. Meeting state reporting 


1 


2 


3 


4 


requirements 

3. Guiding internal resource 


1 


2 


3 


4 


allocation decisions 
4. Guiding undergraduate academic 


1 


2 


3 


4 


program improvement 
5. Improving the achievement of 


1 


2 


3 


4 


undergraduate students 
6. Improving faculty instructional 


1 


2 


3 


4 


performance 

7. Other (briefly describe): 


1 


2 


3 


4 



C. Administrative and Governance Activities 

Institutions have introduced a variety of administrative or governance activities that address or promote student 
assessment Does your institution engage in any of the following activities? (check all that apply): 



1 . 

2 . 

3. 

4. 

5. 

6 . 
7. 



Annual presidential or other institution-wide initiatives, forums or seminars on student assessment 
Rewards or incentives for academic and student affairs administrators who promote use of student 
assessment in their unit 

Incentives for academic units to use student assessment information in their evaluation and 
improvement efforts 

Student assessment workshops for academic and student affairs administrators 
Board of trustees committee that addresses student assessment 
Faculty governance committee that addresses student assessment issues 
^Student representation on student assessment committees 



D. Support for Student Assessment 

Use the scale below to rate the degree to which various groups within your institution support undergraduate student 
assessment activities (circle fine number for each item): 

Very Somewhat Neutral, Very 

Uosuppoxtive Unsupportive Unknown Supportive Supportive 



1 . Board of trustees 1 2 

2. Chief executive officer 1 2 

3. Academic affairs administrators 1 2 

4. Student affairs administrators 1 2 

5. Faculty governance 1 2 

6. Students 1 2 



3 

3 

3 

3 

3 

3 



4 

4 

4 

4 

4 

4 



5 

5 

5 

5 

5 

5 
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E. Planning and Coordinating Student Assessment 

1 . Which of the following best describes your institution’s plan or policy for undergraduate student 
assessment? Your institution (check aH that apply): 

a. has a formally adopted institutional plan or policy requiring specified undergraduate student 

assessment activities of all academic units or programs 

b. has a formally adopted plan or policy for undergraduate student assessment in some academic units 

or program areas (e.g. general education or academic majors) 

c. has a formally adopted institutional plan or policy requiring all academic units or programs to 

develop their own undergraduate student assessment plan 

d. has a formally adopted institutional plan or policy stipulating institution-wide activities to be 

conducted by a central committee, office, or officer 

e. has no formal plan or policy but academic units or programs are encouraged to conduct their own 

undergraduate student assessment activities 

f. is currently developing a plan or policy for undergraduate student assessment 

g. does not have an undergraduate student assessment plan or policy (SKIP TO QUESTION E-6) 

2. Is there an institution-wide group (committee, task force, etc.) that is primarily responsible for ongoing 
planning and policy setting for undergraduate student assessment? (check one) : 

a. yes 

b . no (skip to Question E-5) 

3. If yes, who serves on this group? (check nil that apply): 

a. Chief executive officer 

b. Academic affairs administrator(s)/staff 

c. Student affairs administrator(s)/staff 

d. Institutional research administrator(s)/staff 

e. Academic review and evaluation administrator(s)/staff 

f. Student assessment administrator(s)/staff 

g. Faculty 

h. Students 

i. Other 

4. Who has executive responsibility for or who chairs the institution- wide group responsible for the ongoing 
planning or policy-setting process for undergraduate student assessment? (check all that apply): 

a. Academic affairs administrator 

b. Student affairs administrator 

c. Institutional research officer 

d. Academic review and evaluation officer 

e. Student assessment officer (if separate) 

f. Faculty member 

g. Other 
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Who approves any changes in your institution’s plan or policies for undergraduate student assessment? 
(check all that apply): 

a. Board of trustees 

b . Chief executive officer 

c. Chief academic affairs officer 

d. Chief student affairs officer 

e. Institutional research officer 

f. Academic review and evaluation officer 

g. Student assessment officer 

j . Student government 

h. Academic senate or other faculty committee(s) 

i . Faculty union (If YOUR FACULTY ARE NOT UNIONIZED, CHECK HERE ), 

k. Other 

Who has operational responsibility for your institution’s day-to-day undergraduate student assessment 
activities (e.g., instrument development, data collection, analysis, and reporting)? (check all that apply): 

a. Academic affairs administrator 

b. Student affairs administrator 

c. Institutional research officer 

d. Academic review and evaluation officer 

e. Student assessment officer 

f. Faculty members) 

g . Other 

h. No one (SKIP TO question E8) 

To whom does the individual with operational responsibility for day-to-day student assessment activities 
directly report? (check onek 

a. Chief executive officer 

b . Chief academic officer 

c . Chief student affairs officer 

d. Institutional research officer 

e. Academic review and evaluation officer 

f. Other 

Is there an office which provides faculty consultation in using student assessment for instructional 
improvement or curriculum development? (check onek 

a. yes _ b. no 

If yes, what is the name of the office? 



Evaluating Your Institution’s Student Assessment Plan or Process 
Has your institution evaluated its undergraduate student assessment process? (check onek 

a. yes, with a formal evaluation 

b. yes, with an informal evaluation 

c. currently developing evaluation plans (SKIP TO SECTION HI) 

d. not currently evaluating or planning to evaluate assessment process (SKIP TO SECTION HI) 
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2. In evaluating your institution’s student assessment process, which of the following elements of that 
process were reviewed? (check all that apply): 

a. your student assessment plan and policies 

b. the structure and responsibility for student assessment 

c. achievement of your institution’s intended objectives for student assessment 

d- reliability and validity of student assessment instruments and methods 

e. quality of data analysis 

f* use of student assessment information in institutional decision-making 

g* the problems encountered while conducting student assessment activities 

h. comparison of the costs and benefits of student assessment 



IIL_ External Influences on Institutional Student Assessment Activities 



A . state Role (FOR STATE-FUNDED INSTITUTIONS ONLY; ALL OTHERS SKIP TO QUESTION III B-l) 

1 . Was your state’s plan/requirement for student assessment primarily developed (check onefi : 

a. by state-level officials 

b. through joint consultation between state officials and institutional representatives 

c. no statewide plan or requirement for student assessment exists (SKIP TO QUESTION m. B- 1 ) 

2. State requirements for student assessment (check all that apply): 

a. were an important reason for your institution to initiate undergraduate student assessment 

b. have increased your institution’s involvement in undergraduate student assessment 

c. have not been a factor in your institution’s undergraduate student assessment activities 

d. have been a negative influence on your institution’s undergraduate student assessment activities 

3. Your state’s reporting requirements include (check all that apply): 

a. evidence that a student assessment plan is in place 

b. measurement of state-mandated student performance indicators 

c. institutionally-devised student performance indicators 

d. evidence of institutional use of student assessment information 



4. How has your state higher education agency reviewed or evaluated your institution’s undergraduate student 
assessment plan or process after it was implemented? (check all that apply): 



a. reviewed by state officials 

b . reviewed using external reviewers 

c. required an institutional self-review 

d. no post hoc review has occurred (SKIP TO QUESTION B- 1 ) 



5. The state review of your institution’s undergraduate student assessment plan or process included (check all 
that apply): 



a. 

b. 

c. 

d. 

e. 



review of your institution’s student assessment process itself 

comparison of your institution’s student performance record with your past performance 
comparison of your institution’s student performance record with peer institutions 
comparison of your institution’s student performance record with other institutions in your state 
other (briefly describe) 
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B. Regional Accrediting Role in Student Assessment 

1. Has your institution gone through a regional self study accreditation review which required undergraduate 
student assessment? (check one') : 

a. yes b. no 

2 . Regional accreditation agency requirements for undergraduate student assessment (check all that apply): 

a. were an important reason for your institution to initiate undergraduate student assessment 

b. have increased your institution’s involvement in undergraduate student assessment 

c. have not been a factor in your institution’s undergraduate student assessment activities 

d have been a negative influence on your institution’s undergraduate student assessment activities 

3. Your institution’s regional accreditation agency requires (check ail that apply): 

a. evidence that a student assessment plan or process is in place 

b. intended institutional uses of student assessment information 

c. results of student assessment 

d evidence of actual institutional use of student assessment information 

e. unfamiliar with regional accreditation requirements for student assessment 

C. External Sources of Support for Assessment 

1 . Has your institution received external grants to improve undergraduate student assessment practices from any 
of the following? (check Ml that apply): 

_ a. FIPSE 

b. other federal agencies (please identify): 

c. a state incentive program 

d private foundations or corporate sources (please identify): 

e. no known external grants received 

2. Has your institution used any of the following student assessment services offered by the following 
postsecondary organizations? (check Ml services that apply for each type of organization): 



Student Assessment Service Used 



Type of Postsecondary Organization 

a. Professional associations (Institutional* 
disciplinary, or administrative) 



Not used or Consultation Assessment 

not available services conferences 



Training 

workshops 



Publications 
or research 
reports 



b. Regional accrediting association 

c. State-level agency 



d. Consortium of institutions 
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IV. Academic Management Policies and Practices for Student Assessment 



Institutions have a wide array of formally organized policies, activities, and procedures intended to enhance or support 
the collection and use of undergraduate student assessment information. The following policies and practices have 
been identified in many institutions. 

For Questions a through D, indicate whether the following policies or practices exist at your 

INSTITUTION. 



A. Resource Allocation for Student Assessment (check gH that apply): 

1. An explicit operating budget allocation is made to support student assessment 

2. An academic budget process that considers student performance indicators in resource allocation to academic 

units. 

3. An academic budget process that compares academic units on student performance indicators and allocates 

resources competitively. 

4. An academic budget process that rewards academic units for improvement based on their own past student 

performance indicators. 

B. Student Assessment Information System (check gll that apply): 

1. Key student assessment activities have been scheduled into the academic calendar. 

2. A computerized student information system which includes student performance indicators. 

3. A student information system which tracks individual students from application through graduation. 

4. A student assessment database which is integrated with faculty, curricular, and financial databases. 

C. Access to Individual Student Assessment Information (check gll that apply): 

Student assessment information on individual students is available to: 

1. Institutional research, assessment or evaluation professionals 

2. Senior academic administrators 

3. Department chairs or academic program administrators 

4. Student affairs professionals 

5. Faculty advisors 

D. Distribution of Student Assessment Reports and Studies (check gll that apply): 

Student assessment reports and studies or appropriate summaries are regularly distributed to: 

1. Students 

2. Faculty 

3. Academic administrators 

4. Student affairs professionals 

5. Employers 

6. The general public 
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For Questions E through H, use the following scale to indicate the extent to which each of the 
following POLICIES AND PRACTICES exist At YOUR INSTITUTION (Circle one number for each item). 





Not 


Done in 


Done in 


Done in 


Done it 




done 


a few 


some 


many 


most 


E. Student Policies on Student Assessment 


at all 


depts. 


depts. 


depts. 


depts. 


1. Students are required to participate in student 
assessment activities 


1 


2 


3 


4 


5 


2. Incentives are provided to encourage students to 
. participate in student assessment activities 


1 


2 


3 


4 


5 


3. Information regarding the purpose and uses of student 
assessment is provided to students 


1 


2 


3 


4 


5 


4. Students are provided with individual feedback 
regarding their own student performance results 

F. Professional Development 


1 


2 


3 


4 


5 


1. Faculty are required to learn about or receive training on 
student assessment 


1 


2 


3 


4 


5 


2. Funds for faculty to attend or present at professional 
conferences on student assessment are available 


1 


2 


3 


4 


5 


3. Workshops, seminars, or consultative services for 
faculty on the use of student assessment in course 
design or instruction are offered 


1 


2 


3 


4 


5 


4. Assistance for faculty in the form of paid leaves, 
stipends, mini grants or course reduction to improve 
use of student assessment is provided 


1 


2 


3 


4 


5 


5. Workshops and seminars for department chairs, deans, 
and other academic administrators to improve use of 
student assessment in their unit is provided 


1 


2 


3 


4 


5 


6. Student affairs staff are required to learn about or receive 
training related to student assessment 


1 


2 


3 


4 


5 


7. Student assessment workshops for student affairs 
administrators are provided 

G. Faculty Evaluation and Rewards 


1 


2 


3 


4 


5 


1. Faculty evaluation for promotion considers evidence of 
student performance in their classes (not just student 
teaching evaluation) 


1 


2 


3 


4 


5 


2. Faculty evaluation for annual salary and merit increases 
incorporates evidence of student performance 


1 


2 


3 


4 


5 


3. Faculty scholarship on or innovative uses of student 
assessment is considered in promotion, tenure, or salary 
reviews 


1 


2 


3 


4 


5 


4. Faculty willingness to use or to participate in student 
assessment activities is considered in faculty promotion, 
tenure, or salary reviews 


1 


2 


3 


4 


5 


5. Faculty receive public recognition or awards for 
innovative or effective use of student assessment 


1 


2 


3 


4 


5 


6. Faculty hiring process considers experience or skill in 
student assessment 


1 


2 


3 


4 


5 


7. Faculty are encouraged to assess student learning in 


1 


2 


3 


4 


5 



their classes 
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Not 


Done in 


Done in 


Done in 


Done in 




done 


a few 


some 


many 


most 


H. Academic Planning and Review 


at all 


depts. 


depts. 


depts. 


depts. 


Your institution incorporates student performance data into 
the following processes: 


1. Academic department or undergraduate program 


1 


2 


3 


4 


5 


planning or review 


2. General education or core curriculum review 


1 


2 


3 


4 


5 


3. Course-level review and development 


1 


2 


3 


4 


5 


4. Review and planning for student academic support 


1 


2 


3 


4 


5 


services 



V, Impacts of Student_ Assessment 



A. Decision Making 



To what extent has the use of information available from your undergraduate student assessment process influenced 
the following actions? (circle one number for each item): 



No action or 



influence 

Institutional Actions unknown 

1. Revising your undergraduate 1 

academic mission or goals 

2. Designing or reorganizing 1 

academic programs or majors 

3. Designing or reorganizing student 1 

affairs units 

4. Allocating resources to academic 1 

units 

5. Modifying student assessment 1 

plans, policies, or processes 

6. Deciding faculty promotion and 1 

tenure 

7. Deciding faculty salary increases 1 

or rewards (release time, travel 

funds, etc.) 

8. Revising or modifying general 1 

education curriculum 

9. Creating or modifying student out- 1 

of-class learning experiences (e.g. 
internships, service learning) 

10. Creating or modifying distance 1 

learning initiatives 

1 1 . Modifying instructional or 1 

teaching methods 

12. Modifying student academic 1 

support services (e.g. advising, 
tutoring) 



Action taken, 
data not 
influential 


Action taken, 
data somewhat 
influential 


Action taken, 
data very 
influential 


2 


3 


4 


2 


3 


4 


2 


3 


4 


2 


3 


4 


2 


3 


4 


2 


3 


4 


2 


3 


4 


2 


3 


4 


2 


3 


4 


2 


3 


4 


2 


3 


4 


2 


3 


4 
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B. Institutional Impacts 



1 . Affected campus discussions of 1 

undergraduate education 

2. Contributed to faculty 1 

satisfaction 

3. Contributed to faculty interest 1 

in teaching 

4. Led to changes in instructional 1 

or teaching methods used 

5. Contributed to student 1 

satisfaction 

6. Affected student retention or 1 

graduation rates 

7. Affected student grade 1 

performance 

8. Affected student achievement on 1 

external examinations (e.g. . 
professional licensure, GRE) 



External Impacts 



9. Affected student applications or ^r 1 

student acceptance rates : 

10. Affected allocation or share of 1 

state funding 

1 1 . Affected evaluation from 1 

regional accreditation agency 

12. Affected private fund-raising 1 

results 

13. Affected success on grant 1 

applications 

14. Affected communication with 1 

external constituents 

15. Affected institutional reputation 1 

or image 



impact of student 

Monitored, 
positive 
impact 

4 

4 

4 

4 

4 

4 

4 

4 

234 
2 3 4 

2 3 4 

2 3 4 

2 3 4 

2 3 4 

2 3 4 



Have you monitored the following institutional indicators and been able to document the 
assessment information on them? (circle one number for each item): 



Internal Impacts 



Not monitored, 
do not 
know 



Monitored, 

negative 

impact 



Monitored, 
no known 
impact 



2 

2 

2 

2 

2 

2 

2 

2 



3 

3 

3 

3 

3 

3 

3 

3 
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VI. Further Studies - Optional 



This page will be removed from the questionnaire before it is processed and completion of it is optional. However, 
we would like to know more about your institution’s experience with student assessment and we would like to be 
able to respond to you personally with a follow up report. 



Within the next year several institutions will be invited to participate in a more intensive study of the impacts of 
their student assessment practices and policies. Would you be interested in participating in a case study? 

yes 

possibly 

no 



If you are interested, we would appreciate any additional information regarding your student assessment practices that 
you believe would be of interest to other institutions. If you believe your approach to student assessment or its 
impacts are unusual, please describe it briefly (or enclose a report you think captures your experience). 



Please provide your name and address if you aie interested in receiving a personal summary report of this survey. 



Name: 

Tide: 

Institution: 

Address: 



Phone: 

E-Mail: 



Thank you for taking the time to complete this instrument. 
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Appendix II-A 

Advance Letter to Chief Academic Officer 

January 16, 1998 
[inside address] 



Dear [name], 

~his is to advise you that in about a week you will be receiving an important national survey 
regarding institutional support for student assessment. This survey, which is part of a research 
program being conducted at the University of Michigan, is sponsored by the National Center for 
Postsecondary Improvement (NCPI) — one of the U.S. Department of Education funded research and 
ivelopment centers. Your institution’s participation is critical and I want to take this initial 
opportunity to encourage your participation. 

As you know, assessment of student learning and performance has been a central issue for higher 
education institutions for well over a decade. Yet we have no comprehensive picture of the 
approaches to student assessment that institutions are using or the policies and practices that 
institutions are incorporating to support student assessment. More critically, we have less evidence on 
the impacts of student assessment on institutions and on which institutional support policies and 
practices enhance the use and the impacts of student assessment. The research program for which 
this survey is being conducted addresses these important questions and is designed to help academic 
administrators guide student assessment more effectively on their campuses. 

The survey instrument entitled “An Inventory of Institutional Assessment Practices" (ISSA), is 
designed both to inform this study and to .provide your institution with an opportunity to examine 
your own student assessment approach and support patterns. . The instrument itself will require about 
an hour to complete - either by yourself or the person in your office most knowledgeable about 
student assessment at the undergraduate level in your institution. We will provide a summary of the 
survey results to compare with your own institution’s profile. 

We look forward to your participation and to assisting participating institutions in their own 
examination of this important academic management activity. 

Sincerely, 



Marvin W. Peterson 

Professor of Higher Education, University of Michigan 

and Research Director, National Center for Postsecondary Improvement 
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Appendix II-B 

Cover Letter to First Mailing of Inventory 



January 23, 1998 
[inside address] 



Dear [name]. 

About a week ago, I wrote to let you know you would be receiving the enclosed questionnaire, an 
Inventory of Institutional Support for Student Assessment (ISSA), and to encourage your institution’s 
participation. The instrument is a central element in a larger research program examining student 
assessment as part of the National Center for Postsecondary Improvement. NCPI is one of the U.S. 
Department of Education funded research and development centers. 

As I noted in my prior letter, this survey is the first comprehensive national study examining how 
institutions approach or conduct student assessment, what policies and practices they use to support 
student assessment and what uses or impacts student assessment has had. The study is designed not 
only to examine the various approaches and support patterns, but also to try to understand how these 
approaches and patterns are shaped by external forces (e.g., state policy, accreditation, etc.), how they 
influence institutional use of student assessment information and whether they have beneficial 
institutional impacts. The intent is to examine critically how student assessment impacts institutions, 
how to improve it, and how academic administrators can use it to improve the academic performance 
of their institutions. 

The instrument is designed both to provide data for our study and to provide you with an inventory 
of your own institution’s approaches to and patterns of support for student assessment. You will 
receive a summary of the national survey results to compare with your institution’s own profile when 
data analysis is completed this summer. 

Individual institutions will not be identified in any phase of the research and data identifying 
individual institutions will not be released. The identification code on the instrument is for follow up 
and for assuring dissemination of the survey summary to respondents only. 

The survey should be completed by yourself or the person most familiar with student assessment at 
the undergraduate level on your campus - that may be a dean of undergraduate studies, a director of 
student assessment, or an institutional research or academic evaluation officer. The instrument should 
take about an hour to complete and can be returned in the addressed, postage paid envelope. 

Please complete and return the questionnaire by February 13, 1998. Should you have any questions 
or concerns, they can be addressed to the staff on our project by phone at 734-647-2464 or by 
e-mail at <ncpi.proj52@umich.edu>. We appreciate your response and hope that completing this 
inventory will be useful for your institution as well. 

Sincerely, 



Marvin W. Peterson 

Professor of Higher Education, University of Michigan 

and Research Director, National Center for Postsecondary Improvement 
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Appendix II-C 
Reminder Postcard 



Dear Colleague: 

Approximately two weeks ago you received a survey focusing on Institutional 
Support for Student Assessment (ISSA). This survey is part of a national re- 
search project on student assessment being conducted by the National Center 
for Postsecondary Improvement (NCPI). 

To date, we have not received your completed survey. If you have not already 
done so, we encourage you to fill out and return it. If you did not receive a 
copy of this survey or if it has gone astray, please contact our project office by 
phone (734-647-2464) or e-mail <ncpi.proj52@umich.edu> and we will send 
you a new copy. 

Sincerely, 



Marvin W. Peterson, Research Director 
National Center for Postsecondary Improvement 
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Appendix II-D 

Cover Letter to Second Mailing of Inventory 



March 18, 1998 
[inside address] 



Dear [name], 

A few weeks ago, as part of a national research project on student assessment being conducted by the 
National Center for Postsecondary Improvement, I sent you a survey titled an Inventory of 
Institutional Support for Student Assessment. To date, we have not received a completed survey from 
your institution. If you have recently mailed your response, we thank you for participating in this 
project and look forward to receiving your response. In the event you have not yet completed the 
survey or have not received it, I am enclosing another copy for your consideration. 

As I noted in my earlier correspondence, this survey is the first comprehensive national study 
examining how institutions approach student assessment, what policies and practices they use to 
support student assessment and what uses or impacts student assessment has had. The study also tries 
to understand how external forces (e.g., state policy, accreditation requirements) shape institution’s 
approaches to and patterns of support for student assessment, how they influence institutional use of 
student assessment information and whether they have beneficial institutional impacts. The intent of 
this study is to critically examine how student assessment impacts institutions and how academic 
administrators can use it to improve the academic performance of their institutions. 

The survey is designed both to provide data for our study and to provide you with an inventory of 
your own institution’s approaches to and patterns of support for student assessment. You will receive 
a summary of the national survey results to compare with your institution’s own profile when data 
analysis is completed this summer. Individual institutions will not be identified in any phase of the 
research and data identifying individual institutions will not be released. The identification code on 
the instrument is for follow up and for assuring dissemination of the survey summary to respondents 
only. 

The survey should be completed by yourself or the person most familiar with student assessment at 
the undergraduate level on your campus - that may be a dean of undergraduate studies, a director of 
student assessment, or an institutional research officer. The instrument should take about an hour to 
complete and can be returned in the addressed, postage paid envelope. 

Please complete and return the survey by April 10, 1998. Should you have any questions or 
concerns, they can be addressed to the staff on our project by phone at 734-647-2464 or by e-mail at 
<ncpi.proj52@umich.edu>. We appreciate your response and hope that completing this inventory 
will be useful for your institution as well. 

Sincerely, 



Marvin W. Peterson 

Professor of Higher Education, University of Michigan 

and Research Director, National Center for Postsecondary Improvement 
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Appendix II-E 
Thank You Letter 



22 June 1998 
[inside address] 



Dear [name]: 

Thank you for taking the time to complete the Institutional Support for Student Assessment 
Inventory, sponsored by the National Center for Postsecondary Improvement. Our response rate was 
very high and we are looking forward to providing institutions with data on student assessment 
practices and institutional support structures. We will be spending the summer on data analysis and 
will send information to our participating institutions in the fall. 

I especially want to thank you for your interest in participating in our case study phase. This phase of 
our research is very important in creating meaningful models for institutions. As more than 200 
institutions have expressed interest in participating, we are beginning to create criteria on which to 
base our selection of institutions. We may contact you for further information as we progress with 
the selection process. 

Thanks again for your participation thus far and for your willingness to continue working with us. 
Best of luck in your assessment endeavors. 

Sincerely, 



Marvin W. Peterson 
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Appendix IQ 

Institutional Response 
Rates by State 



State 


Institutions 


Received 


Percentage 


Alaska 


6 


4 


67% 


Alabama 


60 


25 


42% 


Arkansas 


41 


25 


61% 


Arizona 


28 


15 


54% 


California 


191 


77 


40% 


Colorado 


34 


16 


47% 


Connecticut 


33 


19 


58% 


Washington DC 


9 


4 


44% 


Delaware 


7 


2 


29% 


Florida 


64 


40 


63% 


Georgia 


69 


42 


61% 


Hawaii 


15 


5 


33% 


Iowa 


51 


33 


65% 


Idaho 


10 


7 


70% 


Illinois 


111 


73 


66% 


Indiana 


58 


38 


66% 


Kansas 


45 


24 


53% 


Kentucky 


42 


22 


52% 


Louisiana 


64 


30 


47% 


Massachusetts 


83 


41 


49% 


Maryland 


45 


20 


44% 


Maine 


21 


10 


48% 


Michigan 


64 


54 


84% 


Minnesota 


60 


34 


57% 


'Missouri 


55 


32 


58% 


Mississippi 


32 


20 


63% 


Montana 


24 


8 


33% 


North Carolina 


110 


60 


55% 


North Dakota 


18 


11 


61% 


Nebraska 


24 


11 


46% 


New Hampshire 


19 


6 


32% 


New Jersey 


47 


27 


57% 


New Mexico 


22 


8 


36% 


Nevada 


7 


5 


71% 


New York 


161 


79 


49% 


Ohio 


98 


65 


66% 


Oklahoma 


38 


25 


66% 


Oregon 


32 


13 


41% 


Pennsylvania 


134 


59 


44% 


Rhode Island 


8 


3 


38% 


South Carolina 


51 


29 


57% 


South Dakota 


18 


10 


56% 


Tennessee 


54 


36 


67% 


Texas 


135 


73 


54% 


Utah 


11 


8 


73% 


Virginia 


65 


46 


71% 


Vermont 


22 


8 


36% 


Washington 


50 


36 


72% 


Wisconsin 


51 


34 


67% 


West Virginia 


23 


14 


61% 


Wyoming 


8 


6 


75% 


Total 


2528 


1392 
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Appendix IV 

Results of 
Factor Analyses 



Appendix IV-A. Factor Analysis of Institutional Approach to Student Assessment 



Questionnaire Section I-A. Extent by Content 


Component 1 


i 


2 


3 


4 


IA1 1 Vocational or professional outcomes 


.887 


.186 


-.220 


.085 


IA12 Further education 


.874 


.131 


-.301 


.027 


IA14 Satisfaction and Experiences after leaving 


.804 


.239 


-.254 


-.033 


IAS Competence in Major 


.209 


.772 


-.223 


-.034 


IA4 General Education competencies 


.129 


.719 


-.311 


.156 


IA3 Higher order skills 


.153 


.694 


-.379 


.035 


IA6 Vocational or professional skill 


.192 


.693 


.015 


.172 


IA8 Student experiences and involvement 


.235 


.206 


-.807 


.042 


IA9 Student satisfaction 


.377 


.189 


-.703 


.190 


IA7 Personal Growth and affective development 


.131 


.319 


-.684 


-.095 


IA13 Civic or social roles 2 


.436 


.187 


-.529 


-.375 


IA10 Student academic progress 2 


.161 


.116 


-.366 


.063 


IA2 Basic college readiness skill 2 


.094 


.267 


.033 


.712 


IA1 Student academic intentions or expectations 2 


.089 


.037 


-.357 


.664 



Note: 1. Bold print indicates factor placement of item. 

2. Item not included in any factor. Maintained as separate item variable. 



Appendix IV-B, Factor Analysis of Institutional Approach to Student Assessment 



Questionnaire Section I-C. Other Student Assessment Methods 


Component 1 


i 


2 


IC9 


Employer interviews or focus groups 


.770 


-.058 


IC8 


Alumni interviews or focus groups 


.737 


-.208 


IC6 


External examination of students 2 


.488 


-.109 


IC7 


Survey or interview withdrawing students 2 


.482 


-.294 


ICS 


Transcript analysis 2 


.430 


-.376 


IC3 


Student performance in capstone courses 


.136 


-.786 


IC2 


Student portfolios or comprehensive projects 


.136 


-.769 


IC1 


Observations of student performance 


.422 


-.555 


IC4 


Student interviews or focus groups 


.476 


-.510 


Note: 


1. Bold print indicates factor placement of item. 







2. Item not included in any factor. Maintained as separate item variable. 
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Appendix IV-C. Factor Analysis of Institutional Approach to Student Assessment 



Questionnaire Section I-E. Student Assessment Studies 


Component 1 


i 


2 


IF? 


Exposure to different teaching methods 


.694 


-.106 


m3 


Patterns of student-faculty interaction 


.686 


-.360 


ffi9 


Classroom, library and/or computing resources 


.675 


-.305 


IE8 


Academic advising patterns 


.654 


-.451 


mi 


Course taking patterns 


.599 


-.377 


IE5 


Residence arrangements 


.148 


-.801 


IE4 


Extra-curricular activities 


.400 


-.732 


IE6 


Financial aid and/or employment 


.429 


-.699 


ffi7 


Admission standards or policies 


.401 


-.625 


Note: 


1. Bold print indicates factor placement of item. 







Appendix IV-D. Factor Analysis of Institutional Support for Student Assessment 



Questionnaire Section II-B. Purpose of Student Assessment 


Component 1 


i 


2 


HB4 Guiding undergraduate academic program improvement 


.850 


-.013 


HB5 Improving achievement of undergraduates 


.837 


.056 


HB6 Improving faculty instructional performance 


.745 


.196 


HB3 Guiding internal resource allocation decisions 


.712 


.401 


HB7 Other purpose 2 


.596 


-.214 


HBl Meeting state reporting requirements 2 


.029 


.854 


HB2 Preparing institutional self-study 2 


.089 


.775 



Note: 1. Bold print indicates factor placement of item. 

2. Item not included in any factor. Maintained as separate item variable. 



BEST cop^ available 
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Appendix IV-E. Factor Analysis of Assessment Management Practices and Policies 



Questionnaire Sections IV-E. Student Policies on Assessment; 
IV-F. Professional Development; 

IV-G. Faculty Evaluation and Rewards; & 

IV-H. Academic Planning and Review 


Component 1 




i 


2 


3 


4 


5 


IVF2 Funds for faculty to attend assessment conferences 


.761 


.101 


.253 


-.300 


.201 


IVF3 Faculty workshops on student assessment 


.756 


.046 


.230 


-.310 


.406 


IVF4 Faculty assistance using student assessment 


.665 


.201 


.071 


-.236 


.155 


IVF5 Department chair student assessment workshops 


.658 


.128 


.213 


-.373 


.520 


IVG6 Hiring process considers assessment skills 2 


.563 


.435 


.197 


-.302 


.114 


IVF1 Faculty required student assessment training 2 


.489 


.111 


.458 


-.361 


.406 


IVG1 Promotion evaluation includes student performance 


.044 


.771 


.165 


-.213 


.141 


IVG2 Salary evaluation includes student performance 


.037 


.758 


.128 


-.208 


.184 


IVG4 Assessment participation considered in evaluation 


.348 


.725 


.254 


-.260 


-.055 


IVG3 Assessment scholarship considered in evaluation 


.330 


.713 


.162 


-.234 


-.035 


IVG5 Faculty assessment uses publicly recognized 


.496 


.504 


.163 


-.259 


.134 


IVE3 Students informed about assessment purposes 


.198 


.094 


.791 


-.382 


.130 


IVE1 Students required to participate 


.135 


.057 


.749 


-.300 


.094 


IVE4 Students provided individual feedback 


.124 


.105 


.747 


-.280 


.136 


IVE2 Encourage student participation with incentives 2 


.104 


.221 


.404 


-.140 


.091 


T "43 Course review incorporates student data 


.306 


.202 


.318 


-.842 


.193 


i rll Program planning incorporates student data 


.241 


.203 


.301 


-.836 


.177 


IVH2 Curriculum review incorporates student data 


.296 


.169 


.355 


-.831 


.182 


IVH4 Student academic support services planning 


.242 


.226 


.303 


-.758 


.371 


IVG7 Faculty encouraged to assess student learning 2 


.459 


.225 


.401 


-.479 


.043 


IVF6 Assessment training required for student affairs 


.273 


.151 


.258 


-.339 


.879 


IVF7 Assessment workshops for student affairs 
administrators 


.409 


.133 


.200 


-.334 


.869 



Note: 1. Bold print indicates factor placement of item. 



2. Item not included in any factor. Maintained as separate item variable. 




378 430 



Appendix IV-F. Factor Analysis of Impacts of Student Assessment 



Questionnaire Section V-A. Decision Making 


Component 1 


1 


2 


VAll 


Modify instruction or teaching methods 


.707 


.264 


VA2 


Design or reorganize academic programs 


.686 


.260 


VA8 


Revise general education curriculum 


.660 


.167 


VA9 


Create out-of-class learning experiences 


.655 


.213 


VA1 


Revise undergraduate academic mission 


.638 


.266 


VA12 


Modify student academic support services 


.635 


.245 


VA5 


Modify student assessment plans or processes 


.604 


.119 


VA3 


Design or reorganize student affairs 


.582 


.285 


VA4 


Allocate resources to academic units 


.569 


.409 


VA10 


Create distance learning initiatives 


.542 


.248 


VA7 


Decide faculty salary increases 


.334 


.903 


VA6 


Decide faculty promotion and tenure 


.353 


.900 


Note: 


1 . Bold print indicates factor placement of item. 







Appendix IV-G. Factor Analysis of Impacts of Student Assessment 



Questionnaire Section V-B. Institutional Impacts 


Component 1 


1 


2 


3 


VB14 


Communication with external constituents 


.785 


.429 


-.347 


VB13 


Success on grant applications 


.765 


.238 


-.387 


VB12 


Private fund raising results 


.753 - 


.242 - • 


-.376 


VB15 


Institutional reputation or image 


.750 


.461 


-.423 


VB9 


Student applications 


.653 


.177 


-.524 


VB10 


Allocation of state funding 


.593 


.111 


-.306 


VB11 


Regional accreditation agency evaluation 


.546 


.399 


-.258 


VB3 


Faculty interest in teaching 


.314 


.832 


-.344 


VB2 


Faculty satisfaction 


.309 


.760 


-.430 


VB1 


Campus discussions of undergraduate education 


.332 


.755 


-.205 


VB4 


Changes in instructional methods used 


.365 


.722 


-.468 


VB7 


Student grade performance 


.389 


.320 


-.877 


VB6 


Student retention or graduation rates 


.472 


.330 


-.865 


VB5 


Student satisfaction 


.416 


.424 


-.731 


VB8 


Student achievement on external examinations 


.434 


.292 


-.726 


Note: 


1. Bold print indicates factor placement of item. 
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Appendix V-A 

Related Reports and Publications 

Peterson, M. W. (1998). Assessing institutional support for student assessment. InT W Banta 
(Ed.), Assessment Update. Vol. 10 (4). San Francisco: Jossey-Bass. 

Peterson, M. W., Dill, D. D., Mets, L. A., & Associates (Bds.). (1997). Planning and 
management for a changing e nvironment: A handbook on redesigning postserrmHary 
institutions San Francisco: Jossey-Bass. 



Peterson, M. W., & Einarson, M. K. (in press). An analytic framework of institutional support 
for student assessment. In J. Smart (Ed.), Higher education: Handbook of theory and research . 
(Vol. XV). New York: Agathon Press. 

Peterson, M. W., Einarson, M. K., Augustine, C. H. (1997). Inventory of institutional support 
for student assessment. National survey instrument. Stanford, CA: Stanford University, . 
National Center for Postsecondary Improvement. 

Peterson, M. W., Einarson, M. K., Trice, A. G., & Nichols, A. R. (1997). Improving 
organizational and administrat ive support for student assessment: A review of the research 
literature. Stanford, CA: Stanford University, National Center for Postsecondary 
Improvement. 



Peterson, M. W., Einarson, M. K., Trice, A. G., & Nichols, A. R. (1997). An analytic 

framework of instituti onal support for student assessment . Stanford, CA: Stanford University, 
National Center for Postsecondary Improvement. 



Peterson, M. W., Mets, L., Dill, D., & Trice, A. (Eds.). (in press). AS HR reader on institutional 
research and planning . Needham, MA: Simon & Schuster. 

Peterson, M. W., & Trice, A. G. (1997) Institutional evaluation in higher education. Module for 
distance education course for Brazilian administrators. Brasilia, Brazil: UNESCO Project at 
Federal University. 

Peterson, M. W. et al. (1998). Improvement to emergence: An organization-environment research 
agenda for a postsecondarv knowledge industry . Stanford, CA: Stanford University, National 
Center for Postsecondary Improvement. 



Appendix V-B 
Related Presentations 



Augustine, C., Cole, J., & Peterson, M. W. (1998, November). State policy and institutional 
activities. Paper presented at the Association for the Study of Higher Education Annual 
Conference, Miami, FL. 

Peterson, M. W. (1995, November). Institutional support for enhancing student assessment . 
Research proposal for 5.2 section of proposal for National Center for Postsecondary 
Improvement. Stanford, CA.: Stanford University. 

Peterson, M. W. (1995, October). Quality and continuous improvement research on university 
work environments. Presentation at University of Michigan Quality Expo, Ann Arbor, MI. 

Peterson, M. W. (1995, October). Systemic context for transforming teaching and learning. 
Seminar presentation at International Leadership Program for Senior University Administrators 
from Australia and Southeast Asia. University of Michigan, Ann Arbor. 

Peterson, M. W. (1996, May). Enhancing faculty involvement in institutional research: A 
collaborative action research strategy. Paper presented at the Association for Institutional 
Research National Forum, Albuquerque, NM. 

Peterson, M. W. (1997, March). Institutional support for enhancing student assessment and 
performance. Panel presentation on Effects of Institutions, State Policy, and Academic 
Programs on Learning and Assessment at the American Association of Higher Education 
National Conference, Washington, D.C. 

Peterson, M. W. (1997, May). Organizational and administrative environment for student 
assessment. Panel presentation on Environments for Enhancing Student Assessment at the 
American Association of Higher Education National Conference on Assessment, Miami, FL. 

Peterson, M. W., & Augustine, C. (1998, May). Institutional support for student assessment: 
Results of a national survey. Paper presented at the Association for Institutional Research 
Annual Forum, Minneapolis, MN. 

Pfttp-rsnn, M. W. T & Augustine. C. C1998. November). Results of a national survey of 

institutional support for student assessment. Paper presented at the Association for the Study of 
Higher Education Annual Conference, Miami, FL. 

Peterson, M. W., Augustine, C., & Einarson, M. K. (1999, May). Organizational practices 
enhancing the influence of student assessment information in academic decisions. Paper 
presented at the Association for Institutional Research Annual Conference, Seattle, WA. 

Peterson, M. W. and Colleagues (1996-97). Design of distance education course with eight 
modules on Evaluation In Higher Education. Presentation at the UNESCO project for 
Brazilian Administrators, Federal University of Brasilia, Brasilia, Brazil. 

Peterson, M. W., & Einarson, M. K. (1997, May). Institutional support for student assessment: 
Development of a conceptual framework. Paper presented at the Association for Institutional 
Research National Forum, Orlando, FL. 

Peterson, M. W., & Einarson, M. K. (1998, April). Analytic framework of in stitutional support 
for student assessment. Paper presented at the American Educational Research Association 
Annual Conference, San Diego, CA ' 
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Peterson, M. W., & Einarson, M. K. (1998, August). Planning for student assessment: 

Reconciling the interests of internal and external constituents. Paper presented at the Society for 
College and University Planning Annual Conference, Vancouver, B.C., Canada. 

Peterson, M. W., & Einarson, M. K. (1998, September). Management of the learning process: 
Structures and policies to enhance student assessment. Paper presented at the European 
Association for Institutional Research Annual Conference, San Sebastian, Spain. 

Peterson, M. W., Einarson, M. K., & Augustine, C. (1999, April). The influence of institutional 
approaches to and support for student assessment on the improvement of teaching and student 
performance . Paper presented at the American Educational Research Association Annual 
Conference, Montreal, Canada. 
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